Okie. Got it!
Any python pointers (to the fix/workaround) if you could share, that will help…
Folks this looks like it is just header based update (or atleast the headers that nsepython sends is not adequate). Try the following headers and it works for me:
headers={
“accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7”,
“accept-language”: “en-US,en;q=0.9,en-IN;q=0.8,en-GB;q=0.7”,
“cache-control”: “max-age=0”,
“priority”: “u=0, i”,
“sec-ch-ua”: “"Microsoft Edge";v="129", "Not=A?Brand";v="8", "Chromium";v="129"”,
“sec-ch-ua-mobile”: “?0”,
“sec-ch-ua-platform”: “"Windows"”,
“sec-fetch-dest”: “document”,
“sec-fetch-mode”: “navigate”,
“sec-fetch-site”: “none”,
“sec-fetch-user”: “?1”,
“upgrade-insecure-requests”: “1”,
“user-agent”: “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0”
}
s= requests.Session()
output = s.get(“http://nseindia.com”,headers=headers)
res = s.get(“https://www.nseindia.com/api/option-chain-equities?symbol=LT”,
headers = headers,
auth=(),
)
print(res.json())
2 Likes
@Md_Tanveer_Alam Thanks for this. I tried some permutations and combinations of headers earlier, but somehow it did not work.
Finally for me, if I include only below in headers, it works. I think same should work for all.
headers = {#"accept-encoding":"gzip, deflate, br, zstd",
"accept-language":"en-US,en;q=0.9",
"user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/129.0.0.0 Safari/537.36"}
Notice the first line (accept-encoding). This was present earlier in my code, but have commented it now. With this, it works.
It may be good to have complete header as mentioned by you, but does not seem mandatory for now.
Cheers!
1 Like
Thanks man. I tried the header you shared and it works fine for me too.
@abhi_j i did not try to minimize the headers, just plainly copied from what Edge browser sends. So yes it may contain unnecessary elements and yours looks like a minimal version. Thanks
BTW, it seems we all are actively using nsepython and other NSE resources… Do u guys think we should stay connected via Slack, or some other platform? It will help all of us overcome issues we encounter as the upstream changes and we can discuss the solution.
@dexter : If you already have some channel on slack, then we can use that too.
Slight busy but I will do once I do later.
There is already a slack channel Slack
But I will stick in this discourse forum as it helps me to bifurcate things easier.
You can join there and discuss around with other fellows.
1 Like
I will solve this bakchodi today in some pythonic version. The selenium is crashing my server badly as it takes insane memory.
I checked it mildly during market time. From first look it is evident that they have shifted server which different settings of webserver. It does not have the compression.
So I am like 80% sure that they will fix it themselves when they see it. So should I fix their fuck and then unfix when they unfuck or I just wait for them to unfuck it.
Anyways, doing it.
Will hash it when they unfuck it.
Or, I can do a lazy try except which will solve but with a latency.
Any thoughts?
@dexter : I guess the recent change is related to header change (atleast for the function that I was trying to use and was getting error … it was related to fetching the options data for scripts). With the below code, I was above to unblock myself.
Maybe the issue is occurring at other places but I am not aware as I am using Upstox APIs too.
def nsesite_optionchainscrapper(self,script):
if (script == "NIFTY"):
url = "https://www.nseindia.com/api/option-chain-indices?symbol=" + script
else:
url = "https://www.nseindia.com/api/option-chain-equities?symbol=" + script
headers = {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
"accept-language": "en-US,en;q=0.9,en-IN;q=0.8,en-GB;q=0.7",
"cache-control": "max-age=0",
"priority": "u=0, i",
"sec-ch-ua": '"Microsoft Edge";v="129", "Not=A?Brand";v="8", "Chromium";v="129"',
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": '"Windows"',
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"upgrade-insecure-requests": "1",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0"
}
session = requests.Session()
output = session.get("http://nseindia.com",headers=headers)
res = session.get(url, headers = headers, auth=(),)
response = requests.get(url, headers=headers)
# Check if the request was successful
if response.status_code == 200:
# Parse the response as JSON
data = response.json()
return data
else:
print(f"Failed to retrieve data. Status code: {response.status_code}")
2 Likes
Yes see the difference.My code assumes the compression is enabled. Your code doesn’t.
Let me think of remedy this so that it automatically swaps the header in cases like this
Hey,
If I fix it with this loop, it will not work on Amazon servers. [The old code is working fine in Amazon Servers + Google Collab] See Leverage - Unofficed . All modules working fine.
So should I update just the nsepython and leave nsepythonserver?
Tip - For AWS in Bangalore; or Digitalocean in Bangaalore it is working fine.
Alright, the repo has been blessed with your magical header patch! You’ll even spot your name shining proudly in the code now. Take it for a spin and see if everything’s ticking smoothly.
As for me, I’m just here with my trusty nsepythonserver—untouched by this grand experiment—so I’ll leave the beta testing glory to you!
2 Likes
File “\venv\lib\site-packages\nsepythonserver\rahu.py”, line 478, in nse_get_advances_declines
return nsefetch(‘https://www.nseindia.com/api/equity-stockIndices?index=SECURITIES%20IN%20F%26O’)
File “\venv\lib\site-packages\nsepythonserver\rahu.py”, line 31, in nsefetch
output=json.loads(output)
File “C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\json_init_.py”, line 346, in loads
return _default_decoder.decode(s)
File “C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\json\decoder.py”, line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\json\decoder.py”, line 355, in raw_decode
raise JSONDecodeError(“Expecting value”, s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
when using nse_get_advances_declines on nsepythonserver.
nsepython 2.94
nsepythonserver 2.93
is there a suggestion?
You should not use nsepythonserver in windows. its for linux server only. mostly aws/digitalocean/gcollab tested
uninstalled one. fresh install just the nsepython and test
nsepython is working fine with latest, but nsepythonserver giving error in aws. I am using the server version is AWS and gives error.
Use bangalore server. Servers are also blocked locationwise
HI, I am getting JASON error for sometime.
‘’ Catch JSON-related errors and raise as requests.JSONDecodeError [977]"
Can you help me resolve the issue?
Regards,
Share me the entire code and your system’s info like if it windows or linux so that I can troubleshoot.