Recently, we faced an interesting problem to bypass a captcha like the following:
The user needs to drag the blue knob and place it correctly to the dark placeholder to bypass this captcha. When the user places to wrong position, the dark placeholder will be changed to another position and the shape is also changed too.
The system is public facing so we can't ask them to turn off this captcha :-), and the bot we are designing is an unattended bot, so bypassing this captcha is a must.
By using UI Inspector, we can detect the blue knob, but the dark placeholder (on the right) is not an HTML element but is manipulated by image processing, so we can't inspect it. Also the shape of the dark placeholder is changed, so we can't use the image matching techniques.
So we come up with a trial and error approach:
We implement a loop, in each cycle if we still can detect the blue knob (by web automation), we will drag it to a new postion by increasing the position of the blue knob by a configurable StepUnit.
We can speed up the loop by setting a configurable StartingDistance, and in the first cycle we will accelerate the move by adding this StartingDistance.
Finally, we just do testing and finetune to get the best values for StepUnit and StartingDistance.
The solution will look like below in Auteros Studio:
If you want to learn more about the use of Auteros RPA, reach out to our automation experts.
You can reap the benefits of automation immediately without worrying about infrastructure and license restricted now by getting Free Auteros RPA Community Version.