7 ms·

It's an "internet tech" bug. Every developper should know how hard it is to parse textual data vs well defined binary in a secure and fool proof way. Yet every

by meelooo 12y ago

It's an "internet tech" bug. Every developper should know how hard it is to parse textual data vs well defined binary in a secure and fool proof way. Yet every damn internet piece of infrastructure is based on handling textual data, mash it up, pass it around, escape and unescape it in hundreds of stupid formats. No wonder that most security troubles surfacing over the years are some form of abuse of this crazy design flaw: buffer overruns, sql injection, the openssl bug a few month back and now this. Let's go back to sanity and use well defined binary protocols where there is no damn way to send a command by text but only very explicit semantics, and stop the unix way of thinking that text should be more than a human interface. Text should never be used as a command language in between complex programs. Period.

fabulist 12y ago
I really don't think making protocols less understandable by humans will solve anything. This has always been and always will be a hard problem. Consider this quote which I found in The Shellcoder's Handbook: "Wherever terms have a shifting meaning, independent sets of considerations are liable to become complicated together, and reasonings and results are frequently falsified." -- Ada Lovelace We've known about this since literally the beginning; we'll be cursing ourselves over it until the very end. Vulnerabilities are going nowhere.
- meelooo 12y ago
  It's not about making protocols less understandable by humain, it's about recognising that we are programming computers, not humans and that it's time we accept that there should certainly be a way for the humans to interact with the program at some point we should not force the same kind of interactions on the programs themselves. It's much harder to make text and text based commands' parsing and handling secure than it is to use binary protocols in the first place.
  - fabulist 12y ago
    I just disagree. You seem to be saying we should use binary protocols and load them directly into memory Cap'n Proto style. But what if I'm little endian and you're big endian? Parsing happens. Text-based protocols fit well into human's heads, and its the humans that have to do the debugging. I think it would only make the process of finding bugs slower and more complex, and give advantage to the attackers.
nly 12y ago
All very well said, but until tools like Protocol Buffers became popular the tooling for working with custom binary protocols has been pretty dire.
icebraining 12y ago
Please describe specifically how would that have helped. This has nothing to do with parsing text. The problem here is that Apache et all send untrusted data to a process that treats it as code. It wouldn't matter if HTTP was a binary protocol and if bash read a well defined bytecode instead. I mean, look at shellcodes.
- meelooo 12y ago
  You are describing the problem exactly: it all too tempting to pass text around from user input to command line arguments without any way to validate the text data and assume it's ok because it's easy. It's exactly the same arguments that goes in between static and dynamic typing in programming languages: static typing ensures some sort of semantics is respected. If you pass text around, because it's easy and fast, most of the time you will never validate the data and you have no way to ensure that you are not actually handling a bomb. If the protocol was binary there is no way in hell you would be tempted to pass it's data without validation to an external program because you'd have to respect the API and because there would be not way to just send a bunch of commands. The same goes for sql injections, url buffer overflows, etc. Free form text should only be used for actual human textual data and should NEVER be the interface in between programs. It's way too fuzzily defined to serve as a protocol.
  - icebraining 12y ago
    If the protocol was binary there is no way in hell you would be tempted to pass it's data without validation to an external program because you'd have to respect the API and because there would be not way to just send a bunch of commands I assume you're talking about Apache - but Apache had no way of validating the data. The protocol just said "this is a blob from the client", which any binary protocol for the task must be able to handle. Apache had no business validating it, anymore than it should validate any other content - how should it know what makes it valid? Bash, on the other hand, just received that blob and treated it as an executable. It wouldn't matter if the protocol between the server and bash was binary, since it was a valid value as far as the protocol was concerned. The problem here is the hidden channel between Apache and bash, which never actually talk directly to each other (it's through the CGI binary) but still pass data. It has nothing to do with text protocols.
jl6 12y ago
It's not about text vs binary but about well-specified, documented and understood, versus ad-hoc, convention-based, "infinitely extensible", and taught-through-blogs.
TeMPOraL 12y ago
You're getting close, but you missed the essence. It's not about text vs. binary. It's also not about "well-documented" vs. "ad-hoc". It's about preserving semantics. "The Unix Way" means throwing away all semantic data - passing plain strings with no context, which are then parsed and re-parsed in an completely ad-hoc manner, usually with regexp-based shotgun parsers. Note how SQL injections, or XSS attacks are prevented - people stopped stiching strings together and started generating proper instructions through code. User input is sanitized and driven through process that converts it from untrusted string to trusted data structure. Typing SQL queries in a semantic-aware system looks almost the same as stiching strings (thanks to SQL being flat), but now you can't possibly SQL-inject yourself. So in general: stick to text formats or not, but whatever you do, never glue data structures using tools that work on data medium layer, that are not aware of the structure and meaning of data they are operating on. E.g. never glue strings to build SQL queries or HTML code.

by meelooo 12y ago

fabulist 12y ago

I really don't think making protocols less understandable by humans will solve anything. This has always been and always will be a hard problem. Consider this quote which I found in The Shellcoder's Handbook: "Wherever terms have a shifting meaning, independent sets of considerations are liable to become complicated together, and reasonings and results are frequently falsified." -- Ada Lovelace We've known about this since literally the beginning; we'll be cursing ourselves over it until the very end. Vulnerabilities are going nowhere.

meelooo 12y ago
It's not about making protocols less understandable by humain, it's about recognising that we are programming computers, not humans and that it's time we accept that there should certainly be a way for the humans to interact with the program at some point we should not force the same kind of interactions on the programs themselves. It's much harder to make text and text based commands' parsing and handling secure than it is to use binary protocols in the first place.
- fabulist 12y ago
  I just disagree. You seem to be saying we should use binary protocols and load them directly into memory Cap'n Proto style. But what if I'm little endian and you're big endian? Parsing happens. Text-based protocols fit well into human's heads, and its the humans that have to do the debugging. I think it would only make the process of finding bugs slower and more complex, and give advantage to the attackers.

nly 12y ago

All very well said, but until tools like Protocol Buffers became popular the tooling for working with custom binary protocols has been pretty dire.

icebraining 12y ago

Please describe specifically how would that have helped. This has nothing to do with parsing text. The problem here is that Apache et all send untrusted data to a process that treats it as code. It wouldn't matter if HTTP was a binary protocol and if bash read a well defined bytecode instead. I mean, look at shellcodes.

meelooo 12y ago
You are describing the problem exactly: it all too tempting to pass text around from user input to command line arguments without any way to validate the text data and assume it's ok because it's easy. It's exactly the same arguments that goes in between static and dynamic typing in programming languages: static typing ensures some sort of semantics is respected. If you pass text around, because it's easy and fast, most of the time you will never validate the data and you have no way to ensure that you are not actually handling a bomb. If the protocol was binary there is no way in hell you would be tempted to pass it's data without validation to an external program because you'd have to respect the API and because there would be not way to just send a bunch of commands. The same goes for sql injections, url buffer overflows, etc. Free form text should only be used for actual human textual data and should NEVER be the interface in between programs. It's way too fuzzily defined to serve as a protocol.
- icebraining 12y ago
  If the protocol was binary there is no way in hell you would be tempted to pass it's data without validation to an external program because you'd have to respect the API and because there would be not way to just send a bunch of commands I assume you're talking about Apache - but Apache had no way of validating the data. The protocol just said "this is a blob from the client", which any binary protocol for the task must be able to handle. Apache had no business validating it, anymore than it should validate any other content - how should it know what makes it valid? Bash, on the other hand, just received that blob and treated it as an executable. It wouldn't matter if the protocol between the server and bash was binary, since it was a valid value as far as the protocol was concerned. The problem here is the hidden channel between Apache and bash, which never actually talk directly to each other (it's through the CGI binary) but still pass data. It has nothing to do with text protocols.

jl6 12y ago

It's not about text vs binary but about well-specified, documented and understood, versus ad-hoc, convention-based, "infinitely extensible", and taught-through-blogs.

TeMPOraL 12y ago

You're getting close, but you missed the essence. It's not about text vs. binary. It's also not about "well-documented" vs. "ad-hoc". It's about preserving semantics. "The Unix Way" means throwing away all semantic data - passing plain strings with no context, which are then parsed and re-parsed in an completely ad-hoc manner, usually with regexp-based shotgun parsers. Note how SQL injections, or XSS attacks are prevented - people stopped stiching strings together and started generating proper instructions through code. User input is sanitized and driven through process that converts it from untrusted string to trusted data structure. Typing SQL queries in a semantic-aware system looks almost the same as stiching strings (thanks to SQL being flat), but now you can't possibly SQL-inject yourself. So in general: stick to text formats or not, but whatever you do, never glue data structures using tools that work on data medium layer, that are not aware of the structure and meaning of data they are operating on. E.g. never glue strings to build SQL queries or HTML code.