7 ms·

You should upgrade your version of pandas if possible - that's been fixed for a few versions now.

• Grimm1 7y ago
  Oh interesting I'll have to check, we did an upgrade pass a while ago maybe we just didn't upgrade pandas for whatever reason.
  • qwhelan 7y ago
    As mentioned elsewhere in this thread, it's opt-in to avoid breaking existing behavior. But given that ingestion points are easy to identify, it's pretty straightforward to turn on (especially if you have a schema for your inputs): https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html https://pandas.pydata.org/pandas-docs/stable/user_guide/inte...
    • longemen3000 7y ago
      I saw in implementation (CSV parser in Julia) were the sentinel value was randomly assigned at read time (if a value in the input was equal to the sentinel value, change randomly).after parsing, the sentinel value would be converted to the appropriate data type (Julia Missing)
    • Grimm1 7y ago
      That makes sense and thanks for the info and the link. It will be very useful going forward.