Inconsistency in DAP table schema column format

Community Member

Hi @Edina_Tipter and @LeventeHunyadi 

When accessing parquet datasets for submissions & late_policies there seems to be inconsistencies with interpretation from downstream processing jobs for schema on parquet file for decimal vs float64.

Can canvas data team please look into correcting it - currently there is no-format on these columns and leaving it to interpretation of downstream stack to a decimal giving problems rather than to a format: float64. Possibly this would fix parquet files generated from canvas upstream jobs to create parquet files for the downstream consumers stacks to interpret them consistently as float64 format without any ambiguity as rest of all the schema on number has format float64.

dataset: submissions

- points deducted number<no-format> expected number<float64>

dataset: late_policies

- column  missing_submission_deduction  number<no-format>  expected number<float64>

- column late_submission_deduction  number<no-format> expected number<float64>

- column late_submission_minimum_percent  number<no-format> expected number<float64>


Labels (4)