csvclean: Declare a stable Python API

[DeflateAwning]

The docs (for example, https://csvkit.readthedocs.io/en/latest/scripts/csvclean.html) only show info about using the CLI tools. I'm hoping to use this project within a Python project.

Is there a way this can be included, and the CLI methods can be called with Python? If so, it would be awesome if this API's documentation was available in the readthedocs pages.

[jpmckinney]

See https://agate.readthedocs.io/en/latest/

[DeflateAwning]

Hmm that's strange; when I search "csvclean" on there, I don't see any indication that it's linked to this project at all. Maybe more docs explaining the interconnect would be good?

After searching around those docs quite a bit, I still don't see a way to use the "csvclean" functionality of this CLI tool. I stand by my point.

[jpmckinney]

True, csvclean is one of the utilities that does not have any corresponding features in agate.

CSV Kit is not designed to be used as a Python library. Back around version 1.0, there was a major effort to move functionality to agate, which is designed as a Python library. However, not all functionality was moved over.

The remaining functionality in CSV Kit could be organized into a stable API. If we publish the API as part of the docs, then that is a public declaration of a stable API, and we will have to accept the overhead of managing API changes, properly documenting the API, etc. However, there is not a lot of demand (or maintainer availability) to do that. So, I would prefer to leave things as is. For csvclean, you can just read the code which is 100 lines here and then another 150 lines here, and you can call it from your code.

[DeflateAwning]

Would it be okay if we kept this issue open to centralize discussion? Or does the issue belong over in agate?

[jpmckinney]

You can already use from csvkit.cleanup import RowChecker and use it in the same way as it is used in https://github.com/wireservice/csvkit/blob/master/csvkit/utilities/csvclean.py

I'll leave the issue open to see if there is any interest.

[alvaro-osvaldo-tm]

I thought about that, an alternative is to create a 'Builder Pattern' that separate the utility implementation and creates a stable and friendly API, for example.

from csvkit.builders import In2CSV

(In2CSV()
          .input('my-dataset.xls')
          .output('converted.csv')
          .addBom()
          .run())

And even expando to use pipelines as in Bash

from csvkit.builders import In2CSV,CSVGrep

csvGrep = CSVGrep()
csvGrep.column('first-name')

(In2CSV()
          .input('my-dataset.xls')
          .output(  csvGrep  )
          .addBom()
          .run())

Is not a straightforward implementation , but if done well can create an stable and friendly API , and allow the utility to evolve independently from the builder.

if done badly will constantly break and being heavily to maintain.