-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Description
Describe the bug
When using either SchemaDefinition.<T>builder().withPojo() (or directly Schema.AVRO()) time conversions for logical types are registered so POJOs with some java.time objects (not all) are correctly serialized.
However when method SchemaDefinition.<T>builder().withJsonDef() is used instead such registration is not performed. We have to manually register those types for these conversions to work:
import org.apache.pulsar.shade.org.apache.avro.data.TimeConversions;
import org.apache.pulsar.shade.org.apache.avro.generic.GenericData;
import org.apache.pulsar.shade.org.apache.avro.reflect.ReflectData;
private static final GenericData[] GD_INSTANCES = { ReflectData.AllowNull.get(), ReflectData.get(), GenericData.get() };
// Static registration using shaded objects before reading any schema
static {
for (GenericData gd : GD_INSTANCES) {
gd.addLogicalTypeConversion(new TimeConversions.DateConversion());
gd.addLogicalTypeConversion(new TimeConversions.TimeMillisConversion());
gd.addLogicalTypeConversion(new TimeConversions.TimestampMillisConversion());
}
}
Weirdly enough we have found that using a POJO with a java.time.Instant and using an schema with a LogicalType timestamp-millis it is converted to Long and assigned to field in POJO that has type java.lang.Instant (How can this even be possible?).
To Reproduce
Create a POJO with an Instant field.
Produce to a topic using that POJO (and registering the Schema).
On the consumer side use the AVRO schema text with SchemaDefinition.<T>builder().withPojo() to read from the topic. Review what is contained in the read objetcts.
For example. If POJO type is TestPojo we could use:
Schema<TestPojo> schema = AvroSchema.of(SchemaDefinition.<TestPojo>builder().withJsonDef(org.apache.avro.reflect.ReflectData.AllowNull.get().getSchema(TestPojo.class)).build());
And use that Schema to create a consumer.
Expected behavior
That same logical conversions are applied both using withJsonDef() or with withPojo()
Screenshots
Not applicable
Desktop (please complete the following information):
Windows
Additional context
This related to #15858