OneHotEncoder with missing values or Type

[olivierlabayle]

Hello,

I think the following results in the case of missing values (or even just a type being a Union{Missing, Any}) are not expected. No error is thrown but the output is not encoded.

# With a missing value
A = (a=categorical(["a", "b", "a", missing]),)
mach = machine(OneHotEncoder(), A)
fit!(mach)
transform(mach)

# No more missing value but the type of A.a is still a union of missing
A.a[end] = "b"
mach = machine(OneHotEncoder(), A)
fit!(mach)
transform(mach)

Wouldn't it just make sense to skip missing values?

[ablaom]

Although not explicitly documented, missing values are not supported. May not be too difficult to add support, as you suggest.

[ablaom]

Assigned: @Chandu-4444

[Chandu-4444]

I need some help here. The idea is to remove all the missing values before fitting the OneHotEncoder right? To do this, as far as I understood, I'm thinking of doing this (dropping all the missing values) in fit!(match) method. But, I think it is in the MLJBase package? Is it fine if I submit a PR there? Or am I getting this wrong? Thanks!

[olivierlabayle]

Hi Kiran,

I am not sure dropping the missing values is the way to go here, at least no what I intended. In my view there are two options here:

1. Officially not support missing values: Raising an error if a column that needs to be encoded contains missing values, currently it silently fails.
2. Add support (what I think you are trying to do): then I think missing values should just be returned as missing because they may be suppported by downstream algorithms.
3. Of course both options could be implemented and selected based on a hyperparameter.

In any case you don't need to do anything in MLJBase and all the work should happen here. The methods you will have to modify are: fit and transform.

What do you think @ablaom ?

[Frank-III]

I think there are some ways to do it. There are some discussions about OneHotEncoder in python's scikit-learn library, and I think this thread could be helpful.

I think the "all-zero" and "category" are more preferred than encoding missing value as just missing because users would have to deal with the missing value afterward why not impute before putting it into a OneHotEncoder.

Could you talk more about why you want to to encode them as missing @olivierlabayle?

[ablaom]

I think a first step would be to support missing values and propagate them by default, as @olivierlabayle suggests. In any case, I would think imputing missing values (for downstream models that can't handle them) is the job of a secondary imputing transformer, and we have a couple of those already.

But I don't have a strong objection to including some options within OneHotEncoder - they just should not be the default.

[Chandu-4444]

julia> X = (name = categorical(["a", "b", "c", "a", "b", missing]),)

# After some steps ...

(name__a = [1.0, 0.0, 0.0, 1.0, 0.0],
 name__b = [0.0, 1.0, 0.0, 0.0, 0.0],
 name__c = [0.0, 0.0, 1.0, 0.0, 0.0],
 name__missing = [missing, missing, missing, missing],)

I tried working on this. I'm currently able to produce the above output.

[ablaom]

Mmm. Not what I had in mind. I think "propagating this missing values" means something like this:

julia> X = (name = categorical(["a", "b", "c", "a", "b", missing]),)

# After some steps ...

(name__a = [1.0, 0.0, 0.0, 1.0, 0.0, missing],
 name__b = [0.0, 1.0, 0.0, 0.0, 1.0, missing],
 name__c = [0.0, 0.0, 1.0, 0.0, 0.0, missing])

while a non-propagating alternative we might provide is

(name__a = [1.0, 0.0, 0.0, 1.0, 0.0, 0.0],
 name__b = [0.0, 1.0, 0.0, 0.0, 0.0, 0.0],
 name__c = [0.0, 0.0, 1.0, 0.0, 0.0, 0.0],
 name__missing = [0.0., 0.0, 0.0, 0.0, 0.0, 1.0]

Note that in both cases the number of observations (rows) is preserved. In your example the output has one less obervation than the input.�

[ablaom]

The basic case has now been implemented. Will open a new issue regarding further enhancements.