scrape a web made with angular

I’m trying to scrape a web made with Angularjs, using java. I use Selenium and a ChromeDriver to tramping the web, and, to know the next step I use devtools of Chrome.

In a ‘traditional’ website I can guess easily the id of the item or where a button goes, looking in Elements or Source tag but, how can I do it in a web made with angular? I mean, where I can found the id, href property or where a button goes in an angular web? Can I find it using devtools of chrome or I need to install something?

Thanks

Answer

First of all, what your doing is probably illegal.

But to give you the benefit of the doubt, and guess that you are doing to a website that belongs to you but you didn’t write, or don’t have the code, or you have permission from the website owner you have two options.

  1. Extension.

You can use this chrome extension, which lets you inspect your AngularJS in the chrome debugger tools.

Then you can check where the ng-click leads to, and look for that function on the scope.

  1. Console.

Select the element you are interested in inspecting, and since $0 returns the current selected element in the DOM you can write in the console:

angular.element($0).scope()

which will return you an object with all the data on the current scope.

Note that you might have to go up the $parent, to find the function you are looking for.

P.S. if you are looking for how to do the same thing in Angular, you can use the following extension (thanks @user1767316) or in the console ng.probe($0).componentInstance

Leave a Reply

Your email address will not be published. Required fields are marked *