I’m looking for some guidance as I’m having trouble finding any answers when I search, as well as struggling to formulate what I actually want into search terms.
In short, I currently have a Python script that will take an excel spreadsheet containing 2 columns, 1 containing a number, the other containing a string. It then uses Selenium and chromedriver to log in to a website, enter the number from column 1, do some more clicking and then add the note from column 2. It loops through all of the numbers in Column 1 to do this.
The script works perfectly but I need to share it with some colleagues and I figure the best way to do that would be to create a web app using Django that they can upload the excel or csv file to, but I’m struggling to figure out how I would then get the script to run and use the data from the uploaded file.
- When a user uploads the file, how do I then get this to start the Python script?
- How do i then post the results of the script, i.e, whether it successfully added a note for each number, back to the uploader? Is this something that can be done in live time on the web page or would it be better to e-mail the uploader with the results once complete? (I currently have it logging to a .txt file, as well as sending me a Telegram message once it’s completed with details of success/failures.)
Any guidance or a point in the right direction would be greatly appreciated.
Thanks in advance.
I have done 2 similar projects using selenium and requests.
if you insist on selenium there are 2 ways you can do it:
1: Client uploads a file. If the file was valid (input form was valid and you made sure file is not corrupted) you open a selenium web-driver and you have the file and you can open it so running the rest of the script should be possible in that same view.( I don’t recommend this because a lot of things can go wrong and uses a lot of memory if you have a large client base and it takes a lot of time to respond to client and client should not close to webpage to get an answer.)
also you have to configure your web server to keep the connection open.
2: Use a task scheduler. I know django-apscheduler but haven’t used it production.
Client uploads a file. you will save it in database related to your user and add a task to your scheduler then respond to user that file upload was successful.
In that task you will run your script some time later or when possible and after finishing you can save the result or just send an Email to the related user about the result.
If you insist on using selenium the second method is better because you can limit how many selenium web-drivers are open and if all of web-drivers are in use you can delay a task.
Also as I said earlier you can do both of this with requests using sessions and keeping cookies and it would be a lot faster and cheaper.