[Update] NSEPython not working in AWS, Google Cloud and WebServers

dexter · August 30, 2020, 7:46am

They are free to file legal suit lol on a small retailer like me .

jaydave · August 30, 2020, 10:07am

Hahaha

I tried with curl… But seems they blocked that also…

dexter · August 30, 2020, 10:08am

From Friday yes. Now two methods work -

Just get the cookies from Curl session

Or

Json Token using Python

I will update later.

pannet1 · September 4, 2020, 12:10pm

did you try with https://alternativeto.net/software/httpie/about/

dexter · September 6, 2020, 1:53pm

I have really no clue what is it

Kapil_Badlani · October 1, 2020, 11:34pm

Hi Dexter/Aeron,

Hope you are doing well.

Thanks for this wonderful library but I need some help with this. I am able to access it from a local machine but from amazon AWS (AWS Linux/RHEL) servers, I am getting the timeout exception.
Please help me out. Thanks in advance.

Regards,
Kapil Badlani

dexter · October 3, 2020, 9:01am

It is not a problem of Python Request. Nse’s robots.txt has blocked all webservers all together. See here
https://www.nseindia.com/robots.txt

User-agent: *

Disallow: /static/htmls/
Disallow: /static/src/
Disallow: /server/
Disallow: /api/

Sitemap: https://www.nseindia.com/sitemap.xml
Sitemap: https://www.nseindia.com/sitemap-stocks.xml

You can use the curl method and initiate shell commands using Python if you want to do it.

Dev_Singasane · January 1, 2021, 10:30pm

Hi

Thanks for your Youtube videos I was able to code my Trading setup.
Also deployed on AWS but getting timeout error as mentioned

Can you please guide on how to fetch data by curl method? or suggest some alternative.

I am new to coding field

dexter · January 1, 2021, 10:43pm

You don’t need to be master of anything. Google away
… How to use Python to execute a cURL command? - Stack Overflow

Jowhar_Fazal_P · January 25, 2021, 2:26pm

Amazing library and I am using it for my Option Strategy analysis.

But doesn’t work on a server.

Have you got time to work on the CURL option yet?

Can you let me know how it is done, if you dont’ have time to modify the library?

dexter · January 25, 2021, 4:39pm

Daughter disrupting my timeline.

I do not plan to add Curl Method in the library unless I see some other library doing it. It is because - That will increase the load in NSE servers and they will guard up their firewalls causing lots of problems for general scavengers like us like they did with their old website.

Jowhar_Fazal_P · January 31, 2021, 5:51am

OK. Thank you. Looking forward to it.

TheHemantKaushik · February 1, 2021, 4:43pm

Hi Dexter!

Your code is working great on local environment. But it is not working on Google cloud functions even with the same code.

Can you please help? I’m getting timeout exception. I’ve tried with NodeJs api as well. I can post both code if you want.

Thanks in advance!

Divij_Bhutani · February 10, 2021, 5:18am

While working google colab, it throws the following error:

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): www.nseindia.com:443

Can anyone please help?

dexter · May 17, 2021, 9:47pm

It does not allow.

If I make NSEPython compatible for servers, it will not work in windows laptops.
If I make NSEPython compatible for laptops, it will not work in server.

Now, tell me how I will solve this.

dexter · May 17, 2021, 9:49pm

Just mentioned in the above post. I will sooner or later shoot a major update once I finish my work log. I have to do certain changes in our Leverage and Artemis project.

Rohit_Bokade · January 11, 2023, 8:55pm

If I am not wrong request package lets you set useragent in the header write?
So basically can’t we bypas that robot block by setting useragent to something like webkit or something

dexter · January 11, 2023, 9:02pm

Here is a hint

Use subprocess and curl

dexter · April 26, 2023, 9:15pm

Now, this is the major update. Added a new library named “nsepythonserver” that will allow you to run nsepython library from Google Collab and other servers.

Just install nsepythonserver instead of nsepython