4 ms·

Can someone point to a good resource for best practises in protocol design? Am currently faced with this problem and would like to avoid having to reinvent the

Can someone point to a good resource for best practises in protocol design? Am currently faced with this problem and would like to avoid having to reinvent the wheel...

jfoutz 10y ago
Gosh, i don't know of anything off the top of my head. The only real tip i have is to byte length encode variable length requests, because scanning strings sucks. having a frame for messages for messages is nice, stick your fixed with stuff up front, and create fixed with length indicators. It sucks that you waste a little space with 0's, but avoiding the scan is pretty wonderful. GET /foo HTTP/1.1 GET HTTP 01.10 0004/foo Some things need to happen in order, you need to be authenticated before being granted access to resources, but that doesn't necessarily require the client knowing. the state at each step, Traditionally you'd have something like HELO jfoutz hunter2 OK REQEUEST 1 OK <bytes for resource 1> REQUEST 2 OK <bytes for resource 2> but really, the server knows if you're authenticated so i could send HELO jfoutz hunter2 REQUEST 1 REQUEST 1 and then get back either OK OK <bytes for resource 1> OK <bytes for resource 2> or NO NO NO or, whatever. generally it's just going to be a bunch of request/response pairs. sometimes you can't make later requests without actually looking at a response. You don't know what css or images to request until the html is parsed, for example. Usually, you can request a bunch of stuff at once, and it'll work out however it works out.
- userbinator 10y ago
  The only real tip i have is to byte length encode variable length requests, because scanning strings sucks. This is really, really good advice to use whenever possible: it means clients can determine apriori how much data they need to read, and perhaps decide whether the length is valid and/or allocate sufficient memory. One of my annoyances with the traditional "almost-text-based" protocols like HTTP, FTP, SMTP, etc. is that parsing them is not trivial and often you need to keep reading until you hit the delimiter or reach an internal limit. In contrast, "read a length, then read length bytes" makes for simple and efficient implementation. There are certainly cases when the amount of data cannot be determined ahead of time, and in those cases I'd suggest chunked-length-prefixing; delimiters are really a method of last resort.
niftich 10y ago
Do you feel your use-cases can't be met by existing protocols (or toolkits built on top of them) like HTTP, AMQP, MQTT?
chubot 10y ago
Maybe this? https://tools.ietf.org/html/rfc3117 https://tools.ietf.org/html/rfc3117 There is a lot of "philosophy" about protocol design, which seems mostly sound. Although I wonder what happened to the protocol mentioned (BXXP or BEEP). I think HTTP was "good enough" as a universal application protocol? This one also talks about HTTP being wildly successful beyond its original design parameters: https://tools.ietf.org/html/rfc5218 https://tools.ietf.org/html/rfc5218

Can someone point to a good resource for best practises in protocol design? Am currently faced with this problem and would like to avoid having to reinvent the wheel...

jfoutz 10y ago

Gosh, i don't know of anything off the top of my head. The only real tip i have is to byte length encode variable length requests, because scanning strings sucks. having a frame for messages for messages is nice, stick your fixed with stuff up front, and create fixed with length indicators. It sucks that you waste a little space with 0's, but avoiding the scan is pretty wonderful. GET /foo HTTP/1.1 GET HTTP 01.10 0004/foo Some things need to happen in order, you need to be authenticated before being granted access to resources, but that doesn't necessarily require the client knowing. the state at each step, Traditionally you'd have something like HELO jfoutz hunter2 OK REQEUEST 1 OK <bytes for resource 1> REQUEST 2 OK <bytes for resource 2> but really, the server knows if you're authenticated so i could send HELO jfoutz hunter2 REQUEST 1 REQUEST 1 and then get back either OK OK <bytes for resource 1> OK <bytes for resource 2> or NO NO NO or, whatever. generally it's just going to be a bunch of request/response pairs. sometimes you can't make later requests without actually looking at a response. You don't know what css or images to request until the html is parsed, for example. Usually, you can request a bunch of stuff at once, and it'll work out however it works out.

userbinator 10y ago
The only real tip i have is to byte length encode variable length requests, because scanning strings sucks. This is really, really good advice to use whenever possible: it means clients can determine apriori how much data they need to read, and perhaps decide whether the length is valid and/or allocate sufficient memory. One of my annoyances with the traditional "almost-text-based" protocols like HTTP, FTP, SMTP, etc. is that parsing them is not trivial and often you need to keep reading until you hit the delimiter or reach an internal limit. In contrast, "read a length, then read length bytes" makes for simple and efficient implementation. There are certainly cases when the amount of data cannot be determined ahead of time, and in those cases I'd suggest chunked-length-prefixing; delimiters are really a method of last resort.

niftich 10y ago

Do you feel your use-cases can't be met by existing protocols (or toolkits built on top of them) like HTTP, AMQP, MQTT?

chubot 10y ago

Maybe this? https://tools.ietf.org/html/rfc3117 https://tools.ietf.org/html/rfc3117 There is a lot of "philosophy" about protocol design, which seems mostly sound. Although I wonder what happened to the protocol mentioned (BXXP or BEEP). I think HTTP was "good enough" as a universal application protocol? This one also talks about HTTP being wildly successful beyond its original design parameters: https://tools.ietf.org/html/rfc5218 https://tools.ietf.org/html/rfc5218