rsbadger

7 ms·

1.
Show HN: Storage.to CLI, upload a file, get a link
3 points by rsbadger 8mo ago|0 comments
2.
There and Back Again – An Ecommerce Tale(shoprocket.io)
1 points by rsbadger 4y ago|0 comments
3.
How to setup a secure file server with Nginx(medium.com)
4 points by rsbadger 4y ago|0 comments
4.
by rsbadger 4y ago
Unfortunately not - the links were indexed and shown in Bing search results
5.
by rsbadger 4y ago
I do have a robots.txt to block this directory. But Bing only listens to that for what to crawl, not what to index.
6.
by rsbadger 4y ago
I’m fairly certain they are. My links ended up indexed in Bing search results. The only place they were ever rendered was in private emails to users. Bing should not be indexing that.
7.
by rsbadger 4y ago
Exactly that.
8.
by rsbadger 4y ago
This is very useful information. You’d really hope that private emails would be excluded by default…
9.
by rsbadger 4y ago
The only purpose of this link was to verify that the email address is valid. Once it’s verified, you can login.
10.
by rsbadger 4y ago
Many people (most?) prefer to signup to services by email address. To do so, those email addresses must be verified. How would you verify it without sending them an email link?
11.
by rsbadger 4y ago
You’re right that it was a bit of an oversight on my behalf, as the links were only generated after a verified human user action (signup) I had assumed the 1 time links to their email would be safe. But regardless of the link action, it sho
12.
by rsbadger 4y ago
I would be pretty mortified if browsers were using user browser sessions to scan content and pass it to bingbot…? What about if you’re browsing something local? Or your bank account?
13.
by rsbadger 4y ago
Almost all of them. A good example is Dropbox link you send to someone. I could generate this link to a private file in my Dropbox, email it you, and Bing (may) index it. https://www.dropbox.com/s/vucien2ns8jktga/d
14.
by rsbadger 4y ago
Exactly - seems the safest way is to explicitly block known bots by user agent from even reaching pages you don't want indexed.
15.
by rsbadger 4y ago
You may be seeing a stale version, try this: https://shoprocket.io/robots.txt?bypass=1 (I made a lot of changes today when testing all, including "visit as Bingbot" from their webmaster tools with and without the
16.
by rsbadger 4y ago
Quite possibly…
17.
by rsbadger 4y ago
They were indexed in Bing results, I’ve shared the URL of that in this thread
18.
by rsbadger 4y ago
I don't think that's very surprising for most people, the real takeaway is that not only will Bing read your emails, but they may also index any links you send and serve them in search results.
19.
by rsbadger 4y ago
Exactly. I also noticed Bing had accessed some non microsoft tokens too. Even gmail accounts were affected. I assume some people have connected their gmail account to the outlook client?
20.
by rsbadger 4y ago
Exactly. From what I've heard today it sounds like most apps have an extra step between the email link and the login, usually a JS step, to check for bots. Not adding that was my downfall I think.
21.
by rsbadger 4y ago
AFAIK, most of those "private" links are actually just "unlisted" but they're still public. I'm sure Bing is indexing those too...
22.
by rsbadger 4y ago
Absolutely
23.
by rsbadger 4y ago
Yeh I was a bit reluctant to post that as it doesn't look great for my app! But here's the results: https://www.bing.com/search?q=https%3A%2F%2Fshoprocket.io%2F...
24.
by rsbadger 4y ago
Yeh I think you're right, an extra JS step on the page that the email link leads to would help a lot.
25.
by rsbadger 4y ago
They are now - I didn't think I had to as all the pages are naturally behind a login, it never crossed my mind that Bing would follow email links, let alone index them in search results.
26.
by rsbadger 4y ago
yeh... https://www.bing.com/search?q=https%3A%2F%2Fshoprocket.io%2F...
27.
by rsbadger 4y ago
Scanning something for malware and publishing it in search results seem like 2 completely different things to me...?
28.
by rsbadger 4y ago
It was blocked by robots.txt but Bing chose to ignore it. I even tried "blocking" the URLs in Bing webmaster tools today and this was the response: "Block request denied We found that the URL submitted for block is important
29.
by rsbadger 4y ago
Essentially yes - but "previewing" link and sending that link to Bingbot to crawl and index is another matter. Imagine you send someone a "private" link to a file...Bing sees that and indexes it for the world to see. Not
30.
by rsbadger 4y ago
Exactly what I was thinking - what really worries me is it also seems to have happened to a lot of @gmail users too. I still can't figure out how Bing managed to find email tokens sent to gmail. Maybe users who connected their gmail ac

More ›