6 ms·

Have a look at Uncle Bob's fitness app, IIRC this is written using the red green refactor cycle, in fact I would suspect most of his other code on github will b

Have a look at Uncle Bob's fitness app, IIRC this is written using the red green refactor cycle, in fact I would suspect most of his other code on github will be the same. https://github.com/unclebob/fitnesse https://github.com/unclebob/fitnesse

cgopalan 13y ago
Thanks but fitnesse is also a testing framework and none of the other repos seem to be well known, IMO. So it would be good to see other examples.
- dalke 13y ago
  I believe there are things to learn from reviewing the FitNesse source code. I once looked at FitNesse as an example of TDD. I was, and am, doubtful of that technique as an improved method for good design. I found a number of security holes. For example, the "files" resource was open to arbitrary directory traversal, so I could download any file on the system. (I reported it. It has since been fixed.) One of the files I could download was the configuration file. It contains a hashed password, but the hash algorithm was trivial to break. Within a couple of hours I had a program which would generate a string that generates the same hash value. (This does not appear to have changed.) Schneier once wrote "There are two types of encryption: one that will prevent your sister from reading your diary and one that will prevent your government." The fitnesse.authentication.HashingCipher is of the first type, which is likely good enough for the purpose. However, something like <salt>+MD5 would be at least equally secure, shorter, and easier for others to understand. Once I had a password to upload a file, there was another directory traversal error which would let me place a file anywhere on the file system. (This has since been fixed.) There are still design errors. For example, to delete the file "foo.txt", do a GET request to http://localhost:8080/files?responder=deleteFile&filename=foo.txt http://localhost:8080/files?responder=deleteFile&filename=fo... . While FitNesse itself uses a POST, the handler can take a POST or a GET. This would easily allow someone to construct a malicious page which secretly wipes things from your server. If you are Martin then don't follow this link: http://fitnesse.org/files?responder=deleteFile&filename=header.vm http://fitnesse.org/files?responder=deleteFile&filename=head... ;) It's also still open to cross-site scripting. For example, http://localhost:8080/files?responder=%3Cscript%3Ealert%28%22hi!%22%29%3C/script%3E http://localhost:8080/files?responder=%3Cscript%3Ealert%28%2... gives a "hi!" alert. Thus, if I can get a FitNesse admin to click on a well-crafted link then I can deface the site. My conclusion was that TDD doesn't lead to any better security design than a non-TDD project. If your production code requires security then TDD isn't enough. My question to TDD people is: what other design techniques should one use in order to have a secure design? So far I haven't gotten an answer. I also noticed any number of places where the server, which claims to be an HTTP/1.1 server, doesn't actually follow the HTTP specification. For example, FitNesse does not include a "Date" header in the response, but section 14.18 of the spec says "Origin servers MUST include a Date header field in all responses, except in these cases", none of which include FitNesse. More esoteric, 5.1.2 of the spec says "all HTTP/1.1 servers MUST accept the absoluteURI form in requests, even though HTTP/1.1 clients will only generate them in requests to proxies", but FitNesse assumes the first character of the Request-URI is a "/", so accepts "GET $ HTTP/1.0" as a valid request line. These are not critical details, and my own HTTP servers don't follow the details of the spec. For most people, the only important test is that the browsers can understand it. My point here is only the observation that TDD doesn't give a better mechanism to ensure that the software design meets an external specification.
  - cgopalan 13y ago
    Thanks for your input. It does seem difficult to find projects that have been successful in the public sphere that have solely adopted TDD as a design method (besides junit and fitnesse, which seem to be somewhat simple solutions to simple problems). Strangely, it might be the case that programmers who have used some form of red-green-refactor TDD have realized its shortcomings, and are not eager to present that publicly, and also recognize that it cant be set in stone as the one true method to write correct code. Maybe its time I stopped trying to research more about TDD, and just conclude that what works for writing correct programs is what has always been the workflow: 1. Spend time analyzing the problem. 2. Come up with an algorithm based on similar problems or something that makes sense to the programmer. 3. Ensure that there are tests to verify it. 4. Implement the algorithm(s). 5. Change code and tests iteratively based on more analysis.
    - dalke 13y ago
      You may be interested in an essay I wrote a few years ago. http://dalkescientific.com/writings/diary/archive/2009/12/29/problems_with_tdd.html http://dalkescientific.com/writings/diary/archive/2009/12/29... . "and are not eager to present that publicly" I think http://lizkeogh.com/2012/06/24/beyond-test-driven-development/ http://lizkeogh.com/2012/06/24/beyond-test-driven-developmen... is a good expression of that. The author used TDD as a way to understand how to develop testable software, but then writes "Here’s a confession: I don’t always write my tests first. I often don’t automate scenarios up-front either. And I consider it right for me not to do so." I found that page because I wanted to give an example of "Spike and Stabilize", which is an alternate to TDD more along the lines of what I do. Your question has inspired me to write a follow-up to that 2009 essay. The main point will be that TDD is not that appropriate for non-functional testing. Beck's original definition is that everything can (and should) be structured around TDD. There are any number of quotes which use a weaker point; use TDD plus other techniques, but not TDD alone. For just one of many examples, see http://disciplinedagiledelivery.wordpress.com/2012/10/23/strategies-for-verifying-non-functional-requirements/ http://disciplinedagiledelivery.wordpress.com/2012/10/23/str... . That link is interesting because it highlights the importance of a QA group in agile, but calls it "parallel independent testing" or "test team" rather than the traditional name of "QA".

cgopalan 13y ago

Thanks but fitnesse is also a testing framework and none of the other repos seem to be well known, IMO. So it would be good to see other examples.

dalke 13y ago
I believe there are things to learn from reviewing the FitNesse source code. I once looked at FitNesse as an example of TDD. I was, and am, doubtful of that technique as an improved method for good design. I found a number of security holes. For example, the "files" resource was open to arbitrary directory traversal, so I could download any file on the system. (I reported it. It has since been fixed.) One of the files I could download was the configuration file. It contains a hashed password, but the hash algorithm was trivial to break. Within a couple of hours I had a program which would generate a string that generates the same hash value. (This does not appear to have changed.) Schneier once wrote "There are two types of encryption: one that will prevent your sister from reading your diary and one that will prevent your government." The fitnesse.authentication.HashingCipher is of the first type, which is likely good enough for the purpose. However, something like <salt>+MD5 would be at least equally secure, shorter, and easier for others to understand. Once I had a password to upload a file, there was another directory traversal error which would let me place a file anywhere on the file system. (This has since been fixed.) There are still design errors. For example, to delete the file "foo.txt", do a GET request to http://localhost:8080/files?responder=deleteFile&filename=foo.txt http://localhost:8080/files?responder=deleteFile&filename=fo... . While FitNesse itself uses a POST, the handler can take a POST or a GET. This would easily allow someone to construct a malicious page which secretly wipes things from your server. If you are Martin then don't follow this link: http://fitnesse.org/files?responder=deleteFile&filename=header.vm http://fitnesse.org/files?responder=deleteFile&filename=head... ;) It's also still open to cross-site scripting. For example, http://localhost:8080/files?responder=%3Cscript%3Ealert%28%22hi!%22%29%3C/script%3E http://localhost:8080/files?responder=%3Cscript%3Ealert%28%2... gives a "hi!" alert. Thus, if I can get a FitNesse admin to click on a well-crafted link then I can deface the site. My conclusion was that TDD doesn't lead to any better security design than a non-TDD project. If your production code requires security then TDD isn't enough. My question to TDD people is: what other design techniques should one use in order to have a secure design? So far I haven't gotten an answer. I also noticed any number of places where the server, which claims to be an HTTP/1.1 server, doesn't actually follow the HTTP specification. For example, FitNesse does not include a "Date" header in the response, but section 14.18 of the spec says "Origin servers MUST include a Date header field in all responses, except in these cases", none of which include FitNesse. More esoteric, 5.1.2 of the spec says "all HTTP/1.1 servers MUST accept the absoluteURI form in requests, even though HTTP/1.1 clients will only generate them in requests to proxies", but FitNesse assumes the first character of the Request-URI is a "/", so accepts "GET $ HTTP/1.0" as a valid request line. These are not critical details, and my own HTTP servers don't follow the details of the spec. For most people, the only important test is that the browsers can understand it. My point here is only the observation that TDD doesn't give a better mechanism to ensure that the software design meets an external specification.
- cgopalan 13y ago
  Thanks for your input. It does seem difficult to find projects that have been successful in the public sphere that have solely adopted TDD as a design method (besides junit and fitnesse, which seem to be somewhat simple solutions to simple problems). Strangely, it might be the case that programmers who have used some form of red-green-refactor TDD have realized its shortcomings, and are not eager to present that publicly, and also recognize that it cant be set in stone as the one true method to write correct code. Maybe its time I stopped trying to research more about TDD, and just conclude that what works for writing correct programs is what has always been the workflow: 1. Spend time analyzing the problem. 2. Come up with an algorithm based on similar problems or something that makes sense to the programmer. 3. Ensure that there are tests to verify it. 4. Implement the algorithm(s). 5. Change code and tests iteratively based on more analysis.
  - dalke 13y ago
    You may be interested in an essay I wrote a few years ago. http://dalkescientific.com/writings/diary/archive/2009/12/29/problems_with_tdd.html http://dalkescientific.com/writings/diary/archive/2009/12/29... . "and are not eager to present that publicly" I think http://lizkeogh.com/2012/06/24/beyond-test-driven-development/ http://lizkeogh.com/2012/06/24/beyond-test-driven-developmen... is a good expression of that. The author used TDD as a way to understand how to develop testable software, but then writes "Here’s a confession: I don’t always write my tests first. I often don’t automate scenarios up-front either. And I consider it right for me not to do so." I found that page because I wanted to give an example of "Spike and Stabilize", which is an alternate to TDD more along the lines of what I do. Your question has inspired me to write a follow-up to that 2009 essay. The main point will be that TDD is not that appropriate for non-functional testing. Beck's original definition is that everything can (and should) be structured around TDD. There are any number of quotes which use a weaker point; use TDD plus other techniques, but not TDD alone. For just one of many examples, see http://disciplinedagiledelivery.wordpress.com/2012/10/23/strategies-for-verifying-non-functional-requirements/ http://disciplinedagiledelivery.wordpress.com/2012/10/23/str... . That link is interesting because it highlights the importance of a QA group in agile, but calls it "parallel independent testing" or "test team" rather than the traditional name of "QA".