9 ms·

You cannot simply publicly access private secure links, can you?

420 points by vin10 3y ago

internetter 3y ago
The fundamental issue is that links without any form of access control are presumed private, simply because there is no public index of the available identifiers. Just last month, a story with a premise of discovering AWS account ids via buckets[0] did quite well on HN. The consensus established in the comments is that if you are relying on your account identifier being private as some form of security by obscurity, you are doing it wrong. The same concept applies here. This isn’t a novel security issue, this is just another method of dorking. [0]: https://news.ycombinator.com/item?id=39512896 https://news.ycombinator.com/item?id=39512896
- ta1243 3y ago
  The problem is links leak. In theory a 256 hex-character link (so 1024 bits) is near infinitely more secure than a 32 character username and 32 character password, as to guess it https://site.com/[256chars] https://site.com/[256chars] As there's 2^1024 combinations. You'd never brute force it vs https://site,com/[32chars] https://site,com/[32chars] with a password of [32chars] As there's 2^256 combinations. Again you can't brute force it, but it's more likely than the 2^1024 combinations. Imagine it's https://site,com/[32chars][32chars] https://site,com/[32chars][32chars] instead. But while guessing the former is harder than the latter, URLs leak a lot, far more than passwords.
  - internetter 3y ago
    Dorking is the technique of using public search engine indexes to uncover information that is presumed to be private. It has been used to uncover webcams, credit card numbers, confidential documents, and even spies. The problem is the website administers who are encoding authentication tokens into URL state, not the naive crawlers that find them.
    - shkkmo 3y ago
      It can be OK to put authentication tokens in urls, but those tokens need to (at a bare minimum) have short expirations.
      - knome 3y ago
        >It can be OK to put authentication tokens in urls When would this ever be necessary? URL session tokens have been a bad idea ever since they first appeared. The only things even near to auth tokens I can reasonably see stuffed into a URL are password reset and email confirmation tokens sent to email for one time short expiration use. Outside of that, I don't see any reason for it.
        dylanowen 3y ago
        They're useful for images when you can't use cookies and want the client to easily be able to embed them.
        albert_e 3y ago
        "presigned" URLs[1] are a pretty standard and recommended way of providing users access to upload/download content to Amazon S3 buckets without needing other forms of authentication like IAM credential pair, or STS token, etc Web Applications do utilize this pattern very frequently But as noted i previous comment these do have short expiry times (configurable) so that there is no permanent or long-term risk on the lines of the OP article [1]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-...
        knome 3y ago
        Interesting. I haven't built on s3, and if I did my first instinct would probably have been to gate things through a website. Thanks for sharing your knowledge in that area.
        vin10 3y ago
        You are right about short expiry times but another catch here is that if pre-signed URLs are being leaked in an automated fashion, these services also keep the downloaded content from these URLs around. I found various such examples where links no longer work, but PDFs downloaded from pre-signed URLs were still stored by scanning services. From https://urlscan.io/blog/2022/07/11/urlscan-pro-product-updates-for-q2-2022/ https://urlscan.io/blog/2022/07/11/urlscan-pro-product-updat... > In the process of scanning websites, urlscan.io will sometimes encounter file downloads triggered by the website. If we are able to successfully download the file, we will store it, hash it and make it available for downloading by our customers.
  - 4death4 3y ago
    Passwords are always private. Links are only sometimes private.
    - QuinnyPig 3y ago
      Yup. There’s a reason putting credentials into url parameters is considered dangerous.
    - bachmeier 3y ago
      Well-chosen passwords stored properly are always private. Passwords also tend to have much longer lifetimes than links.
  - masom 3y ago
    You won't find a specific link, but at some point if you generate millions of urls the 1024 bits will start to return values pretty quick through bruteforce. The one link won't be found quickly, but a bunch of links will. You just need to fetch all possibilities and you'll get data.
    - blueflow 3y ago
      1024 bits seems a bit too much for the birthday problem to be a thing. I looked at [1] to do the calculation but (2^1024)! is a number too large for any of my tools. If someone has a math shortcut to test this idea properly... [1] https://en.wikipedia.org/wiki/Birthday_problem#Calculating_the_probability https://en.wikipedia.org/wiki/Birthday_problem#Calculating_t...
      - saagarjha 3y ago
        Stirling’s approximation?
      - Dylan16807 3y ago
        This isn't the birthday problem. That would be the chance of two random links overlapping. The birthday problem scales with n^2, while trying to guess links scales with m * n, number of guesses multiplied by number of links. (Well, before you apply the logistic taper to it. So you wanted an approximation? There you go. Until you get the chance of a hit to be quite high, it's basically equal to guesses * valid links / 2^1024.)
        ta1243 3y ago
        The chance is less than guessing a random 128 bit username and random 128 bit password. And then guessing a completely different username and password on the very next go. You'd get far more return on investment breaking bitcoin wallets. 2^1024 is 10^308 Lets say there are 12 billion links per person, and 8 billion people. That's 100 billion billion, or 10^20 links. 10^20 / 10^308 is zero. Lets say you can test 10 trillion links a second, and started when the big bang happened, you'll have tested 10^30 links so far. The number of links you'll have found so far is zero.
  - Y_Y 3y ago
    > site-comma-com Did you do that just to upset me?
  - hiddencost 3y ago
    No. In theory they are both totally insecure.
  - noahtallen 3y ago
    You can easily rate-limit an authentication attempt, to make brute-forcing account access practically impossible, even for a relatively insecure passwords. How would you do that for the URLs? 5 requests to site.com/[256chars] which all 404 block your IP because you don't have a real link? I guess the security is relying on the fact that only a very a small percentage of the total possible links would be used? Though the likelihood of randomly guessing a link is the same as the % of addressable links used.
    - ummonk 3y ago
      I don’t think you realize how exponentially large the possible combinations of 256 characters would be. In fact it doesn’t need to be anywhere near 256 characters. 64 hexadecimal characters would suffice.
  - ablob 3y ago
    Which alphabet did you take as a basis to reach 2^256 combinations?
    - 1231232131231 3y ago
      Binary?
- bo1024 3y ago
  There's probably details I'm missing, but I think the fundamental issue is that "private" messages between people are presumed private, but actually the platforms we use to send messages do read those messages and access links in them. (I mean messages in a very broad sense, including emails, DMs, pasted links in docs, etc.)
  - internetter 3y ago
    URL scanners are not scanning links contained within platforms that require access control. They haven't guessed your password, and to my knowledge no communications platform is feeding all links behind authentication into one of these public URL scanning databases. As the article acknowledged in the beginning, these links are either exposed as the result of deliberate user action, or misconfigured extensions (that, I might add, are suffering from this exact same misconception). If the actual websites are configured to not use the URL as the authentication state, all this would be avoided
    - tobyjsullivan 3y ago
      The suggestion (in both the article and the parent) is that the platforms themselves are submitting URLs. For example, if I send a link in Discord[0] DM, it might show the recipient a message like “warning: this link is malicious”. How does it know that? It submitted the url to one of these services without your explicit consent. [0] Discord is a hypothetical example. I don’t know if they have this feature. But an increasing number of platforms do.
      - internetter 3y ago
        Where in the article does it suggest this? The two bullet points at the very top of TFA is what I cited to discredit this notion, I read it again and still haven't found anything suggesting the communication platforms are submitting this themselves.
        bombcar 3y ago
- mikepurvis 3y ago
  Bit of a tangent, but I was recently advised by a consultant that pushing private Nix closures to a publicly-accessible S3 bucket was fine since each NAR file has a giant hash in the name. I didn't feel comfortable with it so we ended up going a different route, but I've continued to think about that since how different is it really to have the "secret" be in the URL vs in a token you submit as part of the request for the URL? And I think for me it comes down to the fact that the tokens can be issued on a per-customer basis, and access logs can be monitored to watch for suspicious behaviour and revoke accordingly. Also, as others have mentioned, there's just a different mindset around how much it matters that the list of names of files be kept a secret. On the scale of things Amazon might randomly screw up, accidentally listing the filenames sitting in your public bucket sounds pretty low on the priority list since 99% of their users wouldn't care.
  - radlad 3y ago
    > how different is it really to have the "secret" be in the URL vs in a token you submit as part of the request for the URL? I'm not sure I grok this. Do you mean, for example, sending a token in the POST body, or as a cookie / other header? One disadvantage to having a secret in the URL, versus in a header or body, is that it can appear in web service logs, unless you use a URI fragment. Even then, the URL is visible to the user, and will live in their history and URL bar - from which they may copy and paste it elsewhere.
    - mikepurvis 3y ago
      In this case it's package archives, so they're never accessed from a browser, only from the Nix daemon for binary substitution [1]: https://nixos.wiki/wiki/Binary_Cache https://nixos.wiki/wiki/Binary_Cache
  - nmadden 3y ago
    I wrote about putting secrets in URLs a few years ago: https://neilmadden.blog/2019/01/16/can-you-ever-safely-include-credentials-in-a-url/ https://neilmadden.blog/2019/01/16/can-you-ever-safely-inclu...
- bachmeier 3y ago
  > The fundamental issue is that links without any form of access control are presumed private, simply because there is no public index of the available identifiers. Is there a difference between a private link containing a password and a link taking you to a site where you input the password? Bitwarden Send gives a link that you can hand out to others. It has # followed by a long random string. I'd like to know if there are security issues, because I use it regularly. At least with the link, I can kill it, and I can automatically have it die after a few days. Passwords generally don't work that way.
  - koolba 3y ago
    If there’s a live redirect at least there’s the option to revoke the access if the otherwise public link is leaked. I think that’s what sites like DocuSign do with their public links. You can always regenerate it and have it resent to the intended recipients email, but it expires after some fixed period of time to prevent it from being public forever.
  - 7952 3y ago
    There is a difference in that people intuitively know that entering passwords gives access. Also, it may be different legally as the user could reasonably be expected to know that they are not supposed to access something.
    - BlueTemplar 3y ago
      This condemnation comes to mind : https://arstechnica.com/tech-policy/2014/02/french-journalist-fined-4000-plus-for-publishing-public-documents/ https://arstechnica.com/tech-policy/2014/02/french-journalis...
    - bachmeier 3y ago
      > There is a difference in that people intuitively know that entering passwords gives access. This is a valid argument. However, I'd say that there are two standard practices with links that are a big advantage: giving them a short life, and generating extremely hard to guess URLs. I was a Lastpass customer before their security problems came out. I had many passwords that I made years ago but don't use the service any longer. I moved more into the URL camp at that time. Who knows how many passwords I made 15 or 20 years ago that today are no longer secure.
- fddrdplktrew 3y ago
  legend.
- XorNot 3y ago
  Worked for a company which ran into an S3 bucket naming collision when working with a client - turns out that both sides decided hyphenated-company-name was a good S3 bucket name (my company lost that race obviously). One of those little informative pieces where everytime I do AWS now all the bucket names are usually named <project>-<deterministic hash from a seed value>. If it's really meant to be private then you encrypt the project-name too and provide a script to list buckets with "friendly" names. There's always a weird tradeoff with hosted services where technically the perfect thing (totally random identifiers) is too likely to mostly be an operational burden compared to the imperfect thing (descriptive names).
  - cj 3y ago
    What would encrypting the project name accomplish? Typically if you’re trying to secure a S3 bucket you’ll do that via bucket settings. Many years ago you had to jump through hoops to get things private, but these days there’s a big easy button to make a bucket inaccessible publicly.
    - XorNot 3y ago
      The point is that in some cases the name of the project might itself be considered sensitive in some way, so preventing people testing bucket names by trying to create them helps prevent it, but doesn't completely lock you out of being able to associate the bucket back to its internal name, and allows the names to be deterministic internally - i.e. someone spinning up a test environment is still getting everything marked appropriately, deterministically, and uniquely.
      - brickteacup 3y ago
        > The point is that in some cases the name of the project might itself be considered sensitive in some way probably better to solve that problem by just giving projects easy-to-remember codenames. that's what intelligence agencies and militaries have been doing for years after all
scblock 3y ago
When it comes to the internet if something like this is not protected by anything more than a random string in a URL then they aren't really private. Same story with all the internet connected web cams you can find if you go looking. I thought we knew this already. Why doesn't the "Who is responsible" section even mention this?
- AnotherGoodName 3y ago
  Such links are very useful in an 'it's OK to have security match the use case' type of way. You don't need maximum security for everything. You just want a barrier to widespread sharing in some cases. As an example i hit 'create link share' on a photo in my photo gallery and send someone the link to that photo. I don't want them to have to enter a password. I want the link to show the photo. It's ok for the link to do this. One of the examples they have here is exactly that and it's fine for that use case. In terms of privacy fears the end user could re-share a screenshot at that point anyway even if there was a login. The security matches the use case. The user now has a link to a photo, they could reshare but i trust they won't intentionally do this. The big issue here isn't the links imho. It's the security analysis tools scanning all links a user received via email and making them available to other users in that community. That's more re-sharing than i intended when i sent someone a photo.
  - nonrandomstring 3y ago
    > Such links are very useful in an 'it's OK to have security match the use case' I think you give the most sensible summary. It's about "appropriate and proportional" security for the ease of use trade-off. > the user now has a link to a photo, they could reshare but i trust they won't intentionally do this. Time limits are something missing from most applications to create ephemeral links. Ideally you'd want to choose from something like 1 hour, 12 hours, 24 hours, 72 hours... Just resend if they miss the message and it expires. A good trick is to set a cron job on your VPS to clear /www/tmp/ at midnight every other day. > The big issue here isn't the links imho. It's the security analysis tools scanning all links a user received via email You have to consider anything sent to a recipient of Gmail, Microsoft, Apple - any of the commercial providers - to be immediately compromised. If sending between private domains on unencrypted email then it's immediately compromised by your friendly local intelligence agency. If using PGP or am E2E chat app, assume it _will_ be compromised at the end point eventually, so use an ephemeral link.
deleted 3y ago
[deleted]
victorbjorklund 3y ago
Can someone smarter explain to me what is different between? 1) domain.com/login user: John password: 5 char random password 2) domain.com/12 char random url If we assume both either have the same bruteforce/rate limiting protection (or none at all). Why is 1 more safe than 2?
- deleted 3y ago
  [deleted]
- amanda99 3y ago
  Two things: 1. "Password" is a magic word that makes people less likely to just paste it into anything. 2. Username + passwords are two separate pieces of information that are not normally copy-pasted at the same time or have a canonical way of being stored next to each other.
  - victorbjorklund 3y ago
    1) Make sense. 2) Not sure about that. If someone shares their password with someone else they probably share both the username/email and the password
    - deleted 3y ago
      [deleted]
    - amanda99 3y ago
      Yes, people share usernames and passwords, but there's no single canonical string, like "username=amanda99&password=hithere". For example most of the time when I share user/pass combos, they are in separate messages on Signal. You type them into two different boxes, so you normally copy the username, then the password in separate actions.
      - nightpool 3y ago
        I mean, for HTTP Basic there literally is a single canonical string, and it's not uncommon to see people send you links like https://user:somepasswordhere@example.com https://user:somepasswordhere@example.com. I think the arguments other commenters have made about logging, browser history storage, etc are more convincing
amanda99 3y ago
Off topic: but that links to cloudflare radar which apparently mines data from 1.1.1.1. I was under the impression that 1.1.1.1 did not use user data for any purposes?
- kube-system 3y ago
  CF doesn't sell it or use it for marketing, but the entire way they even got the addresses was because APNIC wanted to study the garbage traffic to 1.1.1.1.
  - amanda99 3y ago
    > CF doesn't sell it or use it for marketing Any source for this? Do you work there? I checked their docs and they say they don't "mine user data", so I wouldn't trust anything they say, at least outside legal documents.
    - kube-system 3y ago
      https://1.1.1.1/dns/ https://1.1.1.1/dns/ > We will never sell your data or use it to target ads. https://developers.cloudflare.com/1.1.1.1/privacy/public-dns-resolver/ https://developers.cloudflare.com/1.1.1.1/privacy/public-dns... > Cloudflare will not sell or share Public Resolver users’ personal data with third parties or use personal data from the Public Resolver to target any user with advertisements. There's a lot of transparency on that page in particular, down to the lists of the fields in the logs.
      - nullc 3y ago
        Ads are among the most benign things someone could use your private information to do. With your personal information: * Parties can engage in price discrimination against you. (You don't shop around? you've got some extra spending money? double prices! Looking up risky activities? high insurance rates!) * Parties can engage in lawfare and blackmail * Parties can influence elections (without advertising, e.g. by gerrymandering or by targeting your participation in elections just based on knowing how you'll vote) * Parties can target people of your race or beliefs for genocide (oh, sorry, I meant to say "lawful orders of governments") Plus the definition of advertising itself is up to interpretation: is sending someone to your door advertising?
boxed 3y ago
Outlook.com leaks links to bing. At work it's a constant attack surface that I have to block by looking at the user agent string. Thankfully they are honest in the user agent!
overstay8930 3y ago
Breaking news: Security by obscurity isn't actually security
- makapuf 3y ago
  Well, I like my password/ssh private key to be kept in obscurity.
  - fiddlerwoaroof 3y ago
    Yeah, I’ve always hated this saying because all security involves something that is kept secret, or “obscure”. Also, obscurity is a valid element of a defense in depth strategy
    - koito17 3y ago
      To play devil's advocate, people discourage "security by obscurity" but not "security with obscurity". That is to say, secrets or "obscurity" as part of a layer in your overall security model isn't what gets contested, it's solely relying on obscure information staying obscure that gets contested. e.g. configuring an sshd accepting password auth and unlimited retries to listen on a non-22 port is "security by obscurity". configuring an sshd to disallow root logins, disallow password authentication, only accept connections from a subset of "trustworthy" IP addresses, and listen on a non-22 port, is "security with obscurity"
    - deleted 3y ago
      [deleted]
    - maxcoder4 3y ago
      The idea behind "security thorough obscurity" is that even if the adversary knows everything about your setup *except the secret keys*, you should be secure. Security through obscurity is any method of protection other than the secret key, like for example: * serving ssh on a random high port * using a custom secret encryption algorithm * hosting an unauthenticated service on a secret subdomain in hope nobody will find out * or with a long directory name Some security thorough obscurity is OK (for example high ports or port knocking help buy time when protecting from a zeroday on the service). It's just that relying only on the security thorough obscurity is bad. In this case, I wouldn't call URLs with embedded key security through obscurity, just a poor key management.
QuercusMax 3y ago
I've always been a bit suspicious of infinite-use "private" links. It's just security thru obscurity. At least when you share a Google doc or something there's an option that explicitly says "anyone with the URL can access this". Any systems I've built that need this type of thing have used Signed URLs with a short lifetime - usually only a few minutes. And the URLs are generally an implementation detail that's not directly shown to the user (although they can probably see them in the browser debug view).
- empath-nirvana 3y ago
  There's functionally no difference between a private link and a link protected by a username and password or an api key, as long as the key space is large enough.
  - ses1984 3y ago
    You can’t revoke an individual user’s access to a hard to guess link.
    - colecut 3y ago
      You can if it's one link per user
      - ses1984 3y ago
        True but if you’re generating one link per user, at what point do you lift up your head and wonder if it wouldn’t be easier to just use authentication?
        jddj 3y ago
        The friction that semi-private links remove is that the recipient doesn't need an account for your service. Any tradeoffs should be viewed in that context.
sbr464 3y ago
All media/photos you upload to a private airtable.com app are public links. No authentication required if you know the url.
- internetter 3y ago
  This is actually fairly common for apps using CDNs – not just airtable. I agree it's potentially problematic
  - blue_green_maps 3y ago
    Yes, this is the case for images uploaded through GitHub comments, I think.
    - eddythompson80 3y ago
      That's not true. There is a JWT token in the url with about 5 minute expiration window.
- andix 3y ago
  There is a dilemma for web developers with images loaded from CDNs or APIs. Regular <img> tags can't set an Authorization header with a token for the request, like you can do with fetch() for API requests. The only possibility is adding a token to the URL or by using cookie authentication. Cookie auth only works if the CDN is on the same domain, even a subdomain can be problematic in many cases.
ttymck 3y ago
Zoom meeting links often have the password appended as a query parameter. Is this link a "private secure" link? Is the link without the password "private secure"?
- bombcar 3y ago
  If the password is randomized for each meeting, the URL link is not so bad, as the meeting will be dead and gone by the time the URL appears elsewhere. But in reality, nobody actually cares and just wants a "click to join" that doesn't require fumbling around - but the previous "just use the meeting ID" was too easily guessed.
  - runeb 3y ago
    Unless its a recurring meeting
godelski 3y ago
There's a clear UX problem here. If you submit a scan it doesn't tell you it is public. There can be a helpful fix: make clear that the scan is public! When submitting a scan it isn't clear, as the article shows. But you have the opportunity to also tell the user that it is public during the scan, which takes time. You also have the opportunity to tell them AFTER the scan is done. There should be a clear button to delist. urlscan.io does a bit better but the language is not quite clear that it means the scan is visible to the public. And the colors just blend in. If something isn't catching to your eye, it might as well be treated as invisible. If there is a way to easily misinterpret language, it will always be misinterpreted. if you have to scroll to find something, it'll never be found.
- heipei 3y ago
  Thanks for your feedback. We show the Submit button on our front page as "Public Scan" to indicate that the scan results will be public. Once the scan has finished it will also contain the same colored banner that says "Public Scan". On each scan result page there is a "Report" button which will immediately de-list the scan result without any interaction from our side. If you have any ideas on how to make the experience more explicit I would be happy to hear it!
  - godelski 3y ago
    I understand, but that is not clear enough. "Public scan" can easily be misinterpreted. Honestly, when I looked at it, I didn't know what it meant. Just looked like idk maybe a mistranslation or something? Is it a scan for the public? Is the scanning done in public? Are the results public? Who knows. Remember that I'm not tech literate and didn't make the project. I'd suggest having two buttons, "public scan" "private scan". That would contextualize the public scan to clarify and when you are scanning is publicly __listed__. And different colors. I think red for "public" would actually be the better choice. Some information could be displayed while scanning. Idk put something like "did you know, using the public scan makes the link visible to others? This helps security researchers. You can delist it by clicking ____" or something like that and do the inverse. It should stand out. There's plenty of time while the scan happens. > On each scan result page there is a "Report" button which will immediately de-list the scan result without any interaction from our side. "Report" is not clear. That makes me think I want to report a problem. Also I think there is a problem with the color scheme. The pallet is nice but at least for myself, it all kinda blends in. Nothing pops. Which can be nice at times, but we want to draw the user to certain things, right? I actually didn't see the report button at first. I actually looked around, scrolled, and then even felt embarrassed when I did find it because it is in an "obvious" spot. One that I even looked at! (so extra embarrassing lol) I think this is exactly one of those problems where when you build a tool everything seems obvious and taken care of. You clearly thought about these issues (far better than most!) but when we put things out into public, we need to see how they get used and where our assumptions miss the mark. I do want to say thank you for making this. I am criticizing not to put you down or dismiss any of the work you've done. You've made a great tool that helps a lot of people. You should feel proud for that! I am criticizing because I want to help make the tool the best tool it can be. Of course these are my opinions. My suggestion would be to look at other opinions as well and see if there are common themes. Godelski isn't right, they're just one of many voices that you have to parse. Keep up the good work :)
r2b2 3y ago
To create private shareable links, store the private part in the hash of the URL. The hash is not transmitted in DNS queries or HTTP requests. Ex. When links.com?token=<secret> is visited, that link will be transmitted and potentially saved (search parameters included) by intermediaries like Cloud Flare. Ex. When links.com#<secret> is visited, the hash portion will not leave the browser. Note: It's often nice to work with data in the hash portion by encoding it as a URL Safe Base64 string. (aka. JS Object ↔ JSON String ↔ URL Safe Base 64 String).
- loginatnine 3y ago
  It's called a fragment FYI!
  - shiomiru 3y ago
    However, window.location calls it "hash". (Also, the query string is "search". I wonder why Netscape named them this way...)
    - loginatnine 3y ago
      Interesting, thanks for the additional info.
  - SilasX 3y ago
    Yeah I was confused by it being referred to as the hash. https://en.wikipedia.org/wiki/URI_fragment?useskin=vector https://en.wikipedia.org/wiki/URI_fragment?useskin=vector
- eterm 3y ago
  If it doesn't leave the browser, how would the server know to serve the private content?
  - jadengeller 3y ago
    Client web app makes POST request. It leaves browser, but not in URL
rpigab 3y ago
Links that are not part of a fast redirect loop will be copied and pasted to be shared because that's what URLs are for, they're universal, they facilitate access to a resource available on a protocol. Access control on anything that is not short-lived must be done outside of the url. When you share links on any channel that is not e2ee, the first agent to access that url is not the person you're sending it to, it is the channel's service, it can be legitimate like Bitwarden looking for favicons to enhance UX, or malicious like FB Messenger crawler that wants to know more about what you are sharing in private messages. Tools like these scanners won't get better UX, because if you explicitly tell users that the scans are public, some of them will think twice about using the service, and this is bad for business, wether they're using it for free or paying a pro license.
qudat 3y ago
Over at pico.sh we are experimenting with an entirely new type of private link by leveraging ssh local forward tunnels: https://pgs.sh/ https://pgs.sh/ We are just getting started but so far we are loving the ergonomics.
AJ007 3y ago
[flagged]
zzz999 3y ago
You can if you use E2EE and not CAs
Terr_ 3y ago
A workaround for this "email-based authentication" problem (without going to a full "make an account with a password" step) is to use temporary one-time codes, so that it doesn't matter if the URL gets accidentally shared. 1. User visits "private" link (Or even a public link where they re-enter their e-mail.) 2. Site e-mails user again with time-limited single-use code. 3. User enters temporary code to confirm ownership of e-mail. 4. Flow proceeds (e.g. with HTTP cookies/session data) with reasonable certainty that the e-mail account owner is involved.
andix 3y ago
A while ago I started to only send password protected links via email. Just with the plaintext password inside the email. This might seem absurd and unsafe on the first glance, but those kind of attacks it can safely prevent. Adding an expiration time is also a good idea, even if it is as long as a few months.
figers 3y ago
We have done one time use query string codes at the end of a URL sent to a user email address or as a text message to allow for this...
kgeist 3y ago
Tried it with the local alternative to Google Disk. Oh my... Immediately found lots of private data, including photos of credit cars (with security codes), scans of IDs, passports... How do you report a site?
rvba 3y ago
Reminds me how some would search for bitcoin wallets via google and kazaa. On a side note, can someome remind me what was the name of the file, I think I have some tiny fraction of a bicoin on an old computer
snthd 3y ago
"private secure links" are indistinguishable from any other link. With HTTP auth links you know the password is a password, so these tools would know which part to hide from public display: > https://username:password@example.com/page https://username:password@example.com/page
- jeroenhd 3y ago
  I think it's quite funny that the URL spec has a section dedicated to authentication, only for web devs to invent ways to pass authentication data in any way but using the built-in security mechanism. I know there are valid reasons (the "are you sure you want to log in as usernam on example.com?" prompt for example) but this is just one of the many ways web dev has built hacks upon hacks where implementing standards would've sufficed. See also: S3 vs WebDAV.
getcrunk 3y ago
What’s wrong with using signed urls and encrypting the object with a unique per user key. It’s adds some cpu time but if it’s encrypted it’s encrypted. * this obviously assumes the objects have a 1-1 mapping with users
dav43 3y ago
A classic one that has a business built on this is pidgeonhole - literally private links for events with people hosting internal company events and users posing private sometimes confidential information. And even banks sign on to these platforms!
BobbyTables2 3y ago
What happened to REST design principles? A GET isn’t supposed to modify server state. That is reserved for POST, PUT, PATCH…
65 3y ago
Well this is interesting. Even quickly searching "docs.google.com" on urlscan.io gets me some spreadsheets with lists of people's names, emails, telephone numbers, and other personal information.
egberts1 3y ago
Sure, you can! This is the part where IP filtering by country and subnet can keep your ports hidden. Also stateful firewall can be crafted to only let certain IP thru after sending a specially-crafted TOTP into a ICMP packet just to get into opening the firewall for your IP.
JensRantil 3y ago
I'm surprised no one has mentioned creating a standard that allows a these sites to check whether it's a private link or not. For example, either a special HTTP header returned when making a HEAD request for the URL, or downloading a file similar to robots.txt that defines globs which are public/private. At least this would (mostly) avoid these links becoming publicly available on the internetz.

420 points by vin10 3y ago

The fundamental issue is that links without any form of access control are presumed private, simply because there is no public index of the available identifiers. Just last month, a story with a premise of discovering AWS account ids via buckets[0] did quite well on HN. The consensus established in the comments is that if you are relying on your account identifier being private as some form of security by obscurity, you are doing it wrong. The same concept applies here. This isn’t a novel security issue, this is just another method of dorking. [0]: https://news.ycombinator.com/item?id=39512896 https://news.ycombinator.com/item?id=39512896

ta1243 3y ago
The problem is links leak. In theory a 256 hex-character link (so 1024 bits) is near infinitely more secure than a 32 character username and 32 character password, as to guess it https://site.com/[256chars] https://site.com/[256chars] As there's 2^1024 combinations. You'd never brute force it vs https://site,com/[32chars] https://site,com/[32chars] with a password of [32chars] As there's 2^256 combinations. Again you can't brute force it, but it's more likely than the 2^1024 combinations. Imagine it's https://site,com/[32chars][32chars] https://site,com/[32chars][32chars] instead. But while guessing the former is harder than the latter, URLs leak a lot, far more than passwords.
- internetter 3y ago
  Dorking is the technique of using public search engine indexes to uncover information that is presumed to be private. It has been used to uncover webcams, credit card numbers, confidential documents, and even spies. The problem is the website administers who are encoding authentication tokens into URL state, not the naive crawlers that find them.
  - shkkmo 3y ago
    It can be OK to put authentication tokens in urls, but those tokens need to (at a bare minimum) have short expirations.
    - knome 3y ago
      >It can be OK to put authentication tokens in urls When would this ever be necessary? URL session tokens have been a bad idea ever since they first appeared. The only things even near to auth tokens I can reasonably see stuffed into a URL are password reset and email confirmation tokens sent to email for one time short expiration use. Outside of that, I don't see any reason for it.
      - dylanowen 3y ago
        They're useful for images when you can't use cookies and want the client to easily be able to embed them.
      - albert_e 3y ago
        "presigned" URLs[1] are a pretty standard and recommended way of providing users access to upload/download content to Amazon S3 buckets without needing other forms of authentication like IAM credential pair, or STS token, etc Web Applications do utilize this pattern very frequently But as noted i previous comment these do have short expiry times (configurable) so that there is no permanent or long-term risk on the lines of the OP article [1]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-...
        knome 3y ago
        Interesting. I haven't built on s3, and if I did my first instinct would probably have been to gate things through a website. Thanks for sharing your knowledge in that area.
        vin10 3y ago
        You are right about short expiry times but another catch here is that if pre-signed URLs are being leaked in an automated fashion, these services also keep the downloaded content from these URLs around. I found various such examples where links no longer work, but PDFs downloaded from pre-signed URLs were still stored by scanning services. From https://urlscan.io/blog/2022/07/11/urlscan-pro-product-updates-for-q2-2022/ https://urlscan.io/blog/2022/07/11/urlscan-pro-product-updat... > In the process of scanning websites, urlscan.io will sometimes encounter file downloads triggered by the website. If we are able to successfully download the file, we will store it, hash it and make it available for downloading by our customers.
- 4death4 3y ago
  Passwords are always private. Links are only sometimes private.
  - QuinnyPig 3y ago
    Yup. There’s a reason putting credentials into url parameters is considered dangerous.
  - bachmeier 3y ago
    Well-chosen passwords stored properly are always private. Passwords also tend to have much longer lifetimes than links.
- masom 3y ago
  You won't find a specific link, but at some point if you generate millions of urls the 1024 bits will start to return values pretty quick through bruteforce. The one link won't be found quickly, but a bunch of links will. You just need to fetch all possibilities and you'll get data.
  - blueflow 3y ago
    1024 bits seems a bit too much for the birthday problem to be a thing. I looked at [1] to do the calculation but (2^1024)! is a number too large for any of my tools. If someone has a math shortcut to test this idea properly... [1] https://en.wikipedia.org/wiki/Birthday_problem#Calculating_the_probability https://en.wikipedia.org/wiki/Birthday_problem#Calculating_t...
    - saagarjha 3y ago
      Stirling’s approximation?
    - Dylan16807 3y ago
      This isn't the birthday problem. That would be the chance of two random links overlapping. The birthday problem scales with n^2, while trying to guess links scales with m * n, number of guesses multiplied by number of links. (Well, before you apply the logistic taper to it. So you wanted an approximation? There you go. Until you get the chance of a hit to be quite high, it's basically equal to guesses * valid links / 2^1024.)
      - ta1243 3y ago
        The chance is less than guessing a random 128 bit username and random 128 bit password. And then guessing a completely different username and password on the very next go. You'd get far more return on investment breaking bitcoin wallets. 2^1024 is 10^308 Lets say there are 12 billion links per person, and 8 billion people. That's 100 billion billion, or 10^20 links. 10^20 / 10^308 is zero. Lets say you can test 10 trillion links a second, and started when the big bang happened, you'll have tested 10^30 links so far. The number of links you'll have found so far is zero.
- Y_Y 3y ago
  > site-comma-com Did you do that just to upset me?
- hiddencost 3y ago
  No. In theory they are both totally insecure.
- noahtallen 3y ago
  You can easily rate-limit an authentication attempt, to make brute-forcing account access practically impossible, even for a relatively insecure passwords. How would you do that for the URLs? 5 requests to site.com/[256chars] which all 404 block your IP because you don't have a real link? I guess the security is relying on the fact that only a very a small percentage of the total possible links would be used? Though the likelihood of randomly guessing a link is the same as the % of addressable links used.
  - ummonk 3y ago
    I don’t think you realize how exponentially large the possible combinations of 256 characters would be. In fact it doesn’t need to be anywhere near 256 characters. 64 hexadecimal characters would suffice.
- ablob 3y ago
  Which alphabet did you take as a basis to reach 2^256 combinations?
  - 1231232131231 3y ago
    Binary?
bo1024 3y ago
There's probably details I'm missing, but I think the fundamental issue is that "private" messages between people are presumed private, but actually the platforms we use to send messages do read those messages and access links in them. (I mean messages in a very broad sense, including emails, DMs, pasted links in docs, etc.)
- internetter 3y ago
  URL scanners are not scanning links contained within platforms that require access control. They haven't guessed your password, and to my knowledge no communications platform is feeding all links behind authentication into one of these public URL scanning databases. As the article acknowledged in the beginning, these links are either exposed as the result of deliberate user action, or misconfigured extensions (that, I might add, are suffering from this exact same misconception). If the actual websites are configured to not use the URL as the authentication state, all this would be avoided
  - tobyjsullivan 3y ago
    The suggestion (in both the article and the parent) is that the platforms themselves are submitting URLs. For example, if I send a link in Discord[0] DM, it might show the recipient a message like “warning: this link is malicious”. How does it know that? It submitted the url to one of these services without your explicit consent. [0] Discord is a hypothetical example. I don’t know if they have this feature. But an increasing number of platforms do.
    - internetter 3y ago
      Where in the article does it suggest this? The two bullet points at the very top of TFA is what I cited to discredit this notion, I read it again and still haven't found anything suggesting the communication platforms are submitting this themselves.
      - bombcar 3y ago
mikepurvis 3y ago
Bit of a tangent, but I was recently advised by a consultant that pushing private Nix closures to a publicly-accessible S3 bucket was fine since each NAR file has a giant hash in the name. I didn't feel comfortable with it so we ended up going a different route, but I've continued to think about that since how different is it really to have the "secret" be in the URL vs in a token you submit as part of the request for the URL? And I think for me it comes down to the fact that the tokens can be issued on a per-customer basis, and access logs can be monitored to watch for suspicious behaviour and revoke accordingly. Also, as others have mentioned, there's just a different mindset around how much it matters that the list of names of files be kept a secret. On the scale of things Amazon might randomly screw up, accidentally listing the filenames sitting in your public bucket sounds pretty low on the priority list since 99% of their users wouldn't care.
- radlad 3y ago
  > how different is it really to have the "secret" be in the URL vs in a token you submit as part of the request for the URL? I'm not sure I grok this. Do you mean, for example, sending a token in the POST body, or as a cookie / other header? One disadvantage to having a secret in the URL, versus in a header or body, is that it can appear in web service logs, unless you use a URI fragment. Even then, the URL is visible to the user, and will live in their history and URL bar - from which they may copy and paste it elsewhere.
  - mikepurvis 3y ago
    In this case it's package archives, so they're never accessed from a browser, only from the Nix daemon for binary substitution [1]: https://nixos.wiki/wiki/Binary_Cache https://nixos.wiki/wiki/Binary_Cache
- nmadden 3y ago
  I wrote about putting secrets in URLs a few years ago: https://neilmadden.blog/2019/01/16/can-you-ever-safely-include-credentials-in-a-url/ https://neilmadden.blog/2019/01/16/can-you-ever-safely-inclu...
bachmeier 3y ago
> The fundamental issue is that links without any form of access control are presumed private, simply because there is no public index of the available identifiers. Is there a difference between a private link containing a password and a link taking you to a site where you input the password? Bitwarden Send gives a link that you can hand out to others. It has # followed by a long random string. I'd like to know if there are security issues, because I use it regularly. At least with the link, I can kill it, and I can automatically have it die after a few days. Passwords generally don't work that way.
- koolba 3y ago
  If there’s a live redirect at least there’s the option to revoke the access if the otherwise public link is leaked. I think that’s what sites like DocuSign do with their public links. You can always regenerate it and have it resent to the intended recipients email, but it expires after some fixed period of time to prevent it from being public forever.
- 7952 3y ago
  There is a difference in that people intuitively know that entering passwords gives access. Also, it may be different legally as the user could reasonably be expected to know that they are not supposed to access something.
  - BlueTemplar 3y ago
    This condemnation comes to mind : https://arstechnica.com/tech-policy/2014/02/french-journalist-fined-4000-plus-for-publishing-public-documents/ https://arstechnica.com/tech-policy/2014/02/french-journalis...
  - bachmeier 3y ago
    > There is a difference in that people intuitively know that entering passwords gives access. This is a valid argument. However, I'd say that there are two standard practices with links that are a big advantage: giving them a short life, and generating extremely hard to guess URLs. I was a Lastpass customer before their security problems came out. I had many passwords that I made years ago but don't use the service any longer. I moved more into the URL camp at that time. Who knows how many passwords I made 15 or 20 years ago that today are no longer secure.
fddrdplktrew 3y ago
legend.
XorNot 3y ago
Worked for a company which ran into an S3 bucket naming collision when working with a client - turns out that both sides decided hyphenated-company-name was a good S3 bucket name (my company lost that race obviously). One of those little informative pieces where everytime I do AWS now all the bucket names are usually named <project>-<deterministic hash from a seed value>. If it's really meant to be private then you encrypt the project-name too and provide a script to list buckets with "friendly" names. There's always a weird tradeoff with hosted services where technically the perfect thing (totally random identifiers) is too likely to mostly be an operational burden compared to the imperfect thing (descriptive names).
- cj 3y ago
  What would encrypting the project name accomplish? Typically if you’re trying to secure a S3 bucket you’ll do that via bucket settings. Many years ago you had to jump through hoops to get things private, but these days there’s a big easy button to make a bucket inaccessible publicly.
  - XorNot 3y ago
    The point is that in some cases the name of the project might itself be considered sensitive in some way, so preventing people testing bucket names by trying to create them helps prevent it, but doesn't completely lock you out of being able to associate the bucket back to its internal name, and allows the names to be deterministic internally - i.e. someone spinning up a test environment is still getting everything marked appropriately, deterministically, and uniquely.
    - brickteacup 3y ago
      > The point is that in some cases the name of the project might itself be considered sensitive in some way probably better to solve that problem by just giving projects easy-to-remember codenames. that's what intelligence agencies and militaries have been doing for years after all

scblock 3y ago

When it comes to the internet if something like this is not protected by anything more than a random string in a URL then they aren't really private. Same story with all the internet connected web cams you can find if you go looking. I thought we knew this already. Why doesn't the "Who is responsible" section even mention this?

AnotherGoodName 3y ago
Such links are very useful in an 'it's OK to have security match the use case' type of way. You don't need maximum security for everything. You just want a barrier to widespread sharing in some cases. As an example i hit 'create link share' on a photo in my photo gallery and send someone the link to that photo. I don't want them to have to enter a password. I want the link to show the photo. It's ok for the link to do this. One of the examples they have here is exactly that and it's fine for that use case. In terms of privacy fears the end user could re-share a screenshot at that point anyway even if there was a login. The security matches the use case. The user now has a link to a photo, they could reshare but i trust they won't intentionally do this. The big issue here isn't the links imho. It's the security analysis tools scanning all links a user received via email and making them available to other users in that community. That's more re-sharing than i intended when i sent someone a photo.
- nonrandomstring 3y ago
  > Such links are very useful in an 'it's OK to have security match the use case' I think you give the most sensible summary. It's about "appropriate and proportional" security for the ease of use trade-off. > the user now has a link to a photo, they could reshare but i trust they won't intentionally do this. Time limits are something missing from most applications to create ephemeral links. Ideally you'd want to choose from something like 1 hour, 12 hours, 24 hours, 72 hours... Just resend if they miss the message and it expires. A good trick is to set a cron job on your VPS to clear /www/tmp/ at midnight every other day. > The big issue here isn't the links imho. It's the security analysis tools scanning all links a user received via email You have to consider anything sent to a recipient of Gmail, Microsoft, Apple - any of the commercial providers - to be immediately compromised. If sending between private domains on unencrypted email then it's immediately compromised by your friendly local intelligence agency. If using PGP or am E2E chat app, assume it _will_ be compromised at the end point eventually, so use an ephemeral link.

deleted 3y ago

[deleted]

victorbjorklund 3y ago

Can someone smarter explain to me what is different between? 1) domain.com/login user: John password: 5 char random password 2) domain.com/12 char random url If we assume both either have the same bruteforce/rate limiting protection (or none at all). Why is 1 more safe than 2?

deleted 3y ago
[deleted]
amanda99 3y ago
Two things: 1. "Password" is a magic word that makes people less likely to just paste it into anything. 2. Username + passwords are two separate pieces of information that are not normally copy-pasted at the same time or have a canonical way of being stored next to each other.
- victorbjorklund 3y ago
  1) Make sense. 2) Not sure about that. If someone shares their password with someone else they probably share both the username/email and the password
  - deleted 3y ago
    [deleted]
  - amanda99 3y ago
    Yes, people share usernames and passwords, but there's no single canonical string, like "username=amanda99&password=hithere". For example most of the time when I share user/pass combos, they are in separate messages on Signal. You type them into two different boxes, so you normally copy the username, then the password in separate actions.
    - nightpool 3y ago
      I mean, for HTTP Basic there literally is a single canonical string, and it's not uncommon to see people send you links like https://user:somepasswordhere@example.com https://user:somepasswordhere@example.com. I think the arguments other commenters have made about logging, browser history storage, etc are more convincing

amanda99 3y ago

Off topic: but that links to cloudflare radar which apparently mines data from 1.1.1.1. I was under the impression that 1.1.1.1 did not use user data for any purposes?

kube-system 3y ago
CF doesn't sell it or use it for marketing, but the entire way they even got the addresses was because APNIC wanted to study the garbage traffic to 1.1.1.1.
- amanda99 3y ago
  > CF doesn't sell it or use it for marketing Any source for this? Do you work there? I checked their docs and they say they don't "mine user data", so I wouldn't trust anything they say, at least outside legal documents.
  - kube-system 3y ago
    https://1.1.1.1/dns/ https://1.1.1.1/dns/ > We will never sell your data or use it to target ads. https://developers.cloudflare.com/1.1.1.1/privacy/public-dns-resolver/ https://developers.cloudflare.com/1.1.1.1/privacy/public-dns... > Cloudflare will not sell or share Public Resolver users’ personal data with third parties or use personal data from the Public Resolver to target any user with advertisements. There's a lot of transparency on that page in particular, down to the lists of the fields in the logs.
    - nullc 3y ago
      Ads are among the most benign things someone could use your private information to do. With your personal information: * Parties can engage in price discrimination against you. (You don't shop around? you've got some extra spending money? double prices! Looking up risky activities? high insurance rates!) * Parties can engage in lawfare and blackmail * Parties can influence elections (without advertising, e.g. by gerrymandering or by targeting your participation in elections just based on knowing how you'll vote) * Parties can target people of your race or beliefs for genocide (oh, sorry, I meant to say "lawful orders of governments") Plus the definition of advertising itself is up to interpretation: is sending someone to your door advertising?

boxed 3y ago

Outlook.com leaks links to bing. At work it's a constant attack surface that I have to block by looking at the user agent string. Thankfully they are honest in the user agent!

overstay8930 3y ago

Breaking news: Security by obscurity isn't actually security

makapuf 3y ago
Well, I like my password/ssh private key to be kept in obscurity.
- fiddlerwoaroof 3y ago
  Yeah, I’ve always hated this saying because all security involves something that is kept secret, or “obscure”. Also, obscurity is a valid element of a defense in depth strategy
  - koito17 3y ago
    To play devil's advocate, people discourage "security by obscurity" but not "security with obscurity". That is to say, secrets or "obscurity" as part of a layer in your overall security model isn't what gets contested, it's solely relying on obscure information staying obscure that gets contested. e.g. configuring an sshd accepting password auth and unlimited retries to listen on a non-22 port is "security by obscurity". configuring an sshd to disallow root logins, disallow password authentication, only accept connections from a subset of "trustworthy" IP addresses, and listen on a non-22 port, is "security with obscurity"
  - deleted 3y ago
    [deleted]
  - maxcoder4 3y ago
    The idea behind "security thorough obscurity" is that even if the adversary knows everything about your setup *except the secret keys*, you should be secure. Security through obscurity is any method of protection other than the secret key, like for example: * serving ssh on a random high port * using a custom secret encryption algorithm * hosting an unauthenticated service on a secret subdomain in hope nobody will find out * or with a long directory name Some security thorough obscurity is OK (for example high ports or port knocking help buy time when protecting from a zeroday on the service). It's just that relying only on the security thorough obscurity is bad. In this case, I wouldn't call URLs with embedded key security through obscurity, just a poor key management.

QuercusMax 3y ago

I've always been a bit suspicious of infinite-use "private" links. It's just security thru obscurity. At least when you share a Google doc or something there's an option that explicitly says "anyone with the URL can access this". Any systems I've built that need this type of thing have used Signed URLs with a short lifetime - usually only a few minutes. And the URLs are generally an implementation detail that's not directly shown to the user (although they can probably see them in the browser debug view).

empath-nirvana 3y ago
There's functionally no difference between a private link and a link protected by a username and password or an api key, as long as the key space is large enough.
- ses1984 3y ago
  You can’t revoke an individual user’s access to a hard to guess link.
  - colecut 3y ago
    You can if it's one link per user
    - ses1984 3y ago
      True but if you’re generating one link per user, at what point do you lift up your head and wonder if it wouldn’t be easier to just use authentication?
      - jddj 3y ago
        The friction that semi-private links remove is that the recipient doesn't need an account for your service. Any tradeoffs should be viewed in that context.

sbr464 3y ago

All media/photos you upload to a private airtable.com app are public links. No authentication required if you know the url.

internetter 3y ago
This is actually fairly common for apps using CDNs – not just airtable. I agree it's potentially problematic
- blue_green_maps 3y ago
  Yes, this is the case for images uploaded through GitHub comments, I think.
  - eddythompson80 3y ago
    That's not true. There is a JWT token in the url with about 5 minute expiration window.
andix 3y ago
There is a dilemma for web developers with images loaded from CDNs or APIs. Regular <img> tags can't set an Authorization header with a token for the request, like you can do with fetch() for API requests. The only possibility is adding a token to the URL or by using cookie authentication. Cookie auth only works if the CDN is on the same domain, even a subdomain can be problematic in many cases.

ttymck 3y ago

Zoom meeting links often have the password appended as a query parameter. Is this link a "private secure" link? Is the link without the password "private secure"?

bombcar 3y ago
If the password is randomized for each meeting, the URL link is not so bad, as the meeting will be dead and gone by the time the URL appears elsewhere. But in reality, nobody actually cares and just wants a "click to join" that doesn't require fumbling around - but the previous "just use the meeting ID" was too easily guessed.
- runeb 3y ago
  Unless its a recurring meeting

godelski 3y ago

There's a clear UX problem here. If you submit a scan it doesn't tell you it is public. There can be a helpful fix: make clear that the scan is public! When submitting a scan it isn't clear, as the article shows. But you have the opportunity to also tell the user that it is public during the scan, which takes time. You also have the opportunity to tell them AFTER the scan is done. There should be a clear button to delist. urlscan.io does a bit better but the language is not quite clear that it means the scan is visible to the public. And the colors just blend in. If something isn't catching to your eye, it might as well be treated as invisible. If there is a way to easily misinterpret language, it will always be misinterpreted. if you have to scroll to find something, it'll never be found.

heipei 3y ago
Thanks for your feedback. We show the Submit button on our front page as "Public Scan" to indicate that the scan results will be public. Once the scan has finished it will also contain the same colored banner that says "Public Scan". On each scan result page there is a "Report" button which will immediately de-list the scan result without any interaction from our side. If you have any ideas on how to make the experience more explicit I would be happy to hear it!
- godelski 3y ago
  I understand, but that is not clear enough. "Public scan" can easily be misinterpreted. Honestly, when I looked at it, I didn't know what it meant. Just looked like idk maybe a mistranslation or something? Is it a scan for the public? Is the scanning done in public? Are the results public? Who knows. Remember that I'm not tech literate and didn't make the project. I'd suggest having two buttons, "public scan" "private scan". That would contextualize the public scan to clarify and when you are scanning is publicly __listed__. And different colors. I think red for "public" would actually be the better choice. Some information could be displayed while scanning. Idk put something like "did you know, using the public scan makes the link visible to others? This helps security researchers. You can delist it by clicking ____" or something like that and do the inverse. It should stand out. There's plenty of time while the scan happens. > On each scan result page there is a "Report" button which will immediately de-list the scan result without any interaction from our side. "Report" is not clear. That makes me think I want to report a problem. Also I think there is a problem with the color scheme. The pallet is nice but at least for myself, it all kinda blends in. Nothing pops. Which can be nice at times, but we want to draw the user to certain things, right? I actually didn't see the report button at first. I actually looked around, scrolled, and then even felt embarrassed when I did find it because it is in an "obvious" spot. One that I even looked at! (so extra embarrassing lol) I think this is exactly one of those problems where when you build a tool everything seems obvious and taken care of. You clearly thought about these issues (far better than most!) but when we put things out into public, we need to see how they get used and where our assumptions miss the mark. I do want to say thank you for making this. I am criticizing not to put you down or dismiss any of the work you've done. You've made a great tool that helps a lot of people. You should feel proud for that! I am criticizing because I want to help make the tool the best tool it can be. Of course these are my opinions. My suggestion would be to look at other opinions as well and see if there are common themes. Godelski isn't right, they're just one of many voices that you have to parse. Keep up the good work :)

r2b2 3y ago

To create private shareable links, store the private part in the hash of the URL. The hash is not transmitted in DNS queries or HTTP requests. Ex. When links.com?token=<secret> is visited, that link will be transmitted and potentially saved (search parameters included) by intermediaries like Cloud Flare. Ex. When links.com#<secret> is visited, the hash portion will not leave the browser. Note: It's often nice to work with data in the hash portion by encoding it as a URL Safe Base64 string. (aka. JS Object ↔ JSON String ↔ URL Safe Base 64 String).

loginatnine 3y ago
It's called a fragment FYI!
- shiomiru 3y ago
  However, window.location calls it "hash". (Also, the query string is "search". I wonder why Netscape named them this way...)
  - loginatnine 3y ago
    Interesting, thanks for the additional info.
- SilasX 3y ago
  Yeah I was confused by it being referred to as the hash. https://en.wikipedia.org/wiki/URI_fragment?useskin=vector https://en.wikipedia.org/wiki/URI_fragment?useskin=vector
eterm 3y ago
If it doesn't leave the browser, how would the server know to serve the private content?
- jadengeller 3y ago
  Client web app makes POST request. It leaves browser, but not in URL

rpigab 3y ago

Links that are not part of a fast redirect loop will be copied and pasted to be shared because that's what URLs are for, they're universal, they facilitate access to a resource available on a protocol. Access control on anything that is not short-lived must be done outside of the url. When you share links on any channel that is not e2ee, the first agent to access that url is not the person you're sending it to, it is the channel's service, it can be legitimate like Bitwarden looking for favicons to enhance UX, or malicious like FB Messenger crawler that wants to know more about what you are sharing in private messages. Tools like these scanners won't get better UX, because if you explicitly tell users that the scans are public, some of them will think twice about using the service, and this is bad for business, wether they're using it for free or paying a pro license.

qudat 3y ago

Over at pico.sh we are experimenting with an entirely new type of private link by leveraging ssh local forward tunnels: https://pgs.sh/ https://pgs.sh/ We are just getting started but so far we are loving the ergonomics.

AJ007 3y ago

[flagged]

zzz999 3y ago

You can if you use E2EE and not CAs

Terr_ 3y ago

A workaround for this "email-based authentication" problem (without going to a full "make an account with a password" step) is to use temporary one-time codes, so that it doesn't matter if the URL gets accidentally shared. 1. User visits "private" link (Or even a public link where they re-enter their e-mail.) 2. Site e-mails user again with time-limited single-use code. 3. User enters temporary code to confirm ownership of e-mail. 4. Flow proceeds (e.g. with HTTP cookies/session data) with reasonable certainty that the e-mail account owner is involved.

andix 3y ago

A while ago I started to only send password protected links via email. Just with the plaintext password inside the email. This might seem absurd and unsafe on the first glance, but those kind of attacks it can safely prevent. Adding an expiration time is also a good idea, even if it is as long as a few months.

figers 3y ago

We have done one time use query string codes at the end of a URL sent to a user email address or as a text message to allow for this...

kgeist 3y ago

Tried it with the local alternative to Google Disk. Oh my... Immediately found lots of private data, including photos of credit cars (with security codes), scans of IDs, passports... How do you report a site?

rvba 3y ago

Reminds me how some would search for bitcoin wallets via google and kazaa. On a side note, can someome remind me what was the name of the file, I think I have some tiny fraction of a bicoin on an old computer

snthd 3y ago

"private secure links" are indistinguishable from any other link. With HTTP auth links you know the password is a password, so these tools would know which part to hide from public display: > https://username:password@example.com/page https://username:password@example.com/page

jeroenhd 3y ago
I think it's quite funny that the URL spec has a section dedicated to authentication, only for web devs to invent ways to pass authentication data in any way but using the built-in security mechanism. I know there are valid reasons (the "are you sure you want to log in as usernam on example.com?" prompt for example) but this is just one of the many ways web dev has built hacks upon hacks where implementing standards would've sufficed. See also: S3 vs WebDAV.

getcrunk 3y ago

What’s wrong with using signed urls and encrypting the object with a unique per user key. It’s adds some cpu time but if it’s encrypted it’s encrypted. * this obviously assumes the objects have a 1-1 mapping with users

dav43 3y ago

A classic one that has a business built on this is pidgeonhole - literally private links for events with people hosting internal company events and users posing private sometimes confidential information. And even banks sign on to these platforms!

BobbyTables2 3y ago

What happened to REST design principles? A GET isn’t supposed to modify server state. That is reserved for POST, PUT, PATCH…

65 3y ago

Well this is interesting. Even quickly searching "docs.google.com" on urlscan.io gets me some spreadsheets with lists of people's names, emails, telephone numbers, and other personal information.

egberts1 3y ago

Sure, you can! This is the part where IP filtering by country and subnet can keep your ports hidden. Also stateful firewall can be crafted to only let certain IP thru after sending a specially-crafted TOTP into a ICMP packet just to get into opening the firewall for your IP.

JensRantil 3y ago

I'm surprised no one has mentioned creating a standard that allows a these sites to check whether it's a private link or not. For example, either a special HTTP header returned when making a HEAD request for the URL, or downloading a file similar to robots.txt that defines globs which are public/private. At least this would (mostly) avoid these links becoming publicly available on the internetz.