v7.1.0

Highlights

• 🐛 Fixed false positive issue with trip_distance_exceeds_shape_distance. Before, when a trip had a shape_dist_traveled value in stop_times.txt but its associated shape had no shape_dist_traveled value in shapes.txt, the trip_distance_exceeds_shape_distance error was triggered. Now null shapes.shape_dist_traveled values are ignored. Thanks to @thelleboid-tsr for fixing this.
• 🐛 Bug fix for performance issue for large feeds on the validator website. Thanks to @takohei for flagging this.
• Modified logs for skipped validators to downgrade from an error to a warning. Thanks to @eibakke.

What's Changed

• docs: Automated update of NOTICE_MIGRATION.md by @github-actions in #2006
• fix: acceptance tests keep dropping errors for missing_required_column by @davidgamez in #2007
• fix: changes logging for skipped validators to atInfo instead of atSevere in printSummary by @eibakke in #2016
• fix: update TripAndShapeDistanceValidator to ignore shape with no shape_dist_traveled (#2018) by @thelleboid-tsr in #2020
• fix: Put back the limit of 50 records per notice type in the html report. by @jcpitre in #2027
• fix: web validator logging by @davidgamez in #2026

New Contributors

• @eibakke made their first contribution in #2016
• @thelleboid-tsr made their first contribution in #2020

Full Changelog: v7.0.0...v7.1.0

v7.0.0

Highlights

🧒 👵 👨 Rules for rider categories in GTFS-Fares

Rider Categories are groups of passengers, like adults, children, seniors, or students, who may get different fare pricing based on their age, status, or needs. As of February this year, it's now possible to describe rider categories in GTFS. We've added validation rules to check the quality of new rider categories data, as well as a new Rider Category Feature that will display in the summary report when this data is provided.

📞🚌 Full list of Flex rules

GTFS Flex is a proposal that ensures riders can discover demand responsive services and how to book them. Flex was officially adopted into GTFS last year. With this validator release, we offer a full list of validation rules for Flex. You can see more under "Flex notices".

Breaking JSON report change: N/A values now display as ""

To help standardize empty strings, empty values are now populated with "" instead of N/A in the generated report.json.

Notice changes

🔴 duplicate_key: Triggers when rider_category_id is not unique in rider_categories.txt.
🔴 foreign_key_violation: Verifies that

• every network_id found in fare_leg_join_rules.txt matches an entry in networks.txt or routes.txt.
• every rider_category_id found in fare_products.txt matches an entry in rider_categories.txt.

🔴 invalid_url: Triggers when eligibility_url in rider_categories.txt is invalid.

🔴 missing_required_field: Triggers when

• from_stop_id or to_stop_id is present in fare_leg_join_rules.txt but the other is omitted
• rider_category_id and/or rider_category_name and/or is_default_fare_category are omitted from rider_categories.txt

🔴 unexpected_enum_value: Triggers when is_default_fare_category is a value other than 0, empty or 1.
⬜ unused_parent_station renamed to unused_station

New notices

🔴 fare_product_with_multiple_default_rider_categories: This notice is generated when a fare product is associated with multiple rider categories that are marked as default.
🟡 transfer_distance_too_large: The transfer distance from stop to stop in transfers.txt is larger than 10 km.
⬜ transfer_distance_above_2_kmThe transfer distance from stop to stop in transfers.txt is larger than 2 km.

Flex notices

Flex new notices

🔴 duplicate_geography_id: ID must be unique across all stops.stop_id, locations.geojson id, and location_groups.location_group_id values.
🔴 forbidden_geography_id: In stop_times.txt, you can have only one of stop_id, location_group_id or location_id defined for given entry.
🔴 missing_stop_times_record: Travel within the same location group or GeoJSON location requires two records in stop_times.txt with the same location_group_id or location_id.
🔴 forbidden_arrival_or_departure_time: The arrival or departure times are provided alongside pickup or drop-off windows in stop_times.txt. This violates GTFS specification, as both cannot coexist for a single stop time record.
🔴 forbidden_continuous_pickup_drop_off: Continuous pickup or drop-off are forbidden when routes.continuous_pickup or routes.continuous_drop_off are 0, 2 or 3 and stop_times.start_pickup_drop_off_window or stop_times.end_pickup_drop_off_window are defined for any trip of this route.
🔴 forbidden_pickup_type: pickup_drop_off_window fields are forbidden when the pickup_type is regularly scheduled (0) or must be coordinated with the driver (3).
🔴 forbidden_drop_off_type: pickup_drop_off_window fields are forbidden when the drop_off_type is regularly scheduled (0).
🔴 invalid_pickup_drop_off_window: The end_pickup_drop_off_window must be strictly later than the start_pickup_drop_off_window.
🔴 missing_pickup_or_drop_off_window: Either the start or end pickup/drop-off window is missing in stop_times.txt.
🟡 missing_pickup_drop_off_booking_rule_id: pickup_booking_rule_id is recommended when pickup_type=2 and drop_off_booking_rule_id is recommended when drop_off_type=2. Currently, this notice is only triggered on feeds when either start_pickup_drop_off_window or end_pickup_drop_off_window is defined, since this recommendation was added to the specification for feeds with GTFS-Flex.

locations.geojson notices

🔴 duplicate_geo_json_key: The key must be unique for each feature in the GeoJSON file.
🔴 geo_json_duplicated_element: Duplicated elements in locations.geojson file.
🔴 overlapping_zone_and_pickup_drop_off_window: Two entities in stop_times.txt with the same trip_id have the same pickup_type or drop_off_type and overlapping pickup/drop-off windows and overlapping zones in locations.geojson.
⬜ geo_json_unknown_element: Unknown elements in locations.geojson file.

Flex notice changes

🔴 foreign_key_violation: Checks that the location_id field in stop_times.txt reference valid entries in the locations.geojson file.
🔴 missing_prior_day_booking_field_value separated into missing_prior_notice_last_day and missing_prior_notice_last_time notices

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility Database in this spreadsheet.

Other notable improvements

• Documentation: Added website documentation on how the feed summary metadata report is extracted, available at https://gtfs-validator.mobilitydata.org/rules.html#summary-metadata.
• Published the Java GUI as a independent jar asset
• Bug: @sylvansson fixed issue where invalid_input_files_in_subfolder was not being triggered in certain valid cases.

New Contributors

• @sylvansson made their first contribution in #1959
• @skalexch made their first contribution in #1981

Full PR List

• docs: Automated update of NOTICE_MIGRATION.md by @github-actions in #1905
• fix: filter out Continuous Stops from flex alert by @cka-y in #1920
• feat: Use a github variable containing 1password secret references by @jcpitre in #1921
• chore: artifact action v3 upgrade by @cka-y in #1926
• chore: upgrade missed e2e upload artifact to v4 by @davidgamez in #1937
• feat: 1822 flex forbidden pickup type & forbidden drop off type by @qcdyx in #1936
• fix: python scripts failing in ubuntu-latest GitHub workflows by @davidgamez in #1944
• feat: pickup and drop off window validator by @cka-y in #1935
• feat: 1820 flex missing stop times record by @qcdyx in #1945
• feat: flex - `overlapping_zone_and_pickup_drop_off_windo...

v6.0.0

Highlights

Partial validation support for Flex

GTFS Flex is a proposal that ensures riders can discover demand responsive services and how to book them. Flex was officially adopted into GTFS this March. With this validator release, we offer partial validation support for feeds that include Flex data. This includes:

• Flex features in summary. If a feed has Flex data, you'll see its Flexible Services in the GTFS Features section of the validation report summary. Flex Features include Booking Rules, Predefined Routes with Deviation, Zone-Based Demand Responsive Services, and Fixed-Stops Demand Responsive Services.

• No more false positives! Before, feeds with Flex features would trigger certain notices when the data was actually valid, such as missing_required_field for a missing stop_id in stop_times.txt when location_group_id or location_id was provided. Now, any error or warning generated for your Flex feed is legitimate and should be fixed.

• Validation for booking_rules.txt. 10 new validation notices have been added to check data from booking_rules.txt

• Validation for polygons in locations.geojson. Checks that each polygon in locations.geojson conforms to the OpenGIS Simple Features Specification, section 6.1.11, as required by GTFS.

We'll continue to add more Flex validation rules in future releases. You can see the full list of validation rules here.

Notice changes

• 🔴 missing_required_file: feed_info.txt is now conditionally required when translations.txt exists. Clarification adopted in google/transit#460.
• 🔴 missing_required_field for transfers.txt: from_stop_id and to_stop_id are no longer required for a transfer_type of 4 or 5. Clarification adopted in google/transit#455.
• 🔴 foreign_key_violation: Now checks if network_id exists in fare_leg_rules.txt but not in routes.txt OR networks.txt. Before, there was a bug that meant network_id had to be included in both routes.txt AND networks.txt.
• 🟡 missing_timepoint_value: Modified to clarify that timepoint values should be set explicitly to 0 or 1, not empty. Clarification adopted into GTFS in google/transit#474. 🟡 missing_recommended_column was also removed as a result of this clarification.

Flex notice changes

• 🔴 missing_required_file: stops.txt is now conditionally required. If locations.geojson is not present, stops.txt is required.
• 🔴 missing_required_field: stop_id is conditionally required in stop_times.txt, in cases where the record does not include location_id or location_group_id.
• 🔴 missing_trip_edge: Only checks stop times records if they do not include start_pickup_dropoff_window or end_pickup_drop_off_window.
• 🔴 decreasing_or_equal_stop_time_distance: Skips stop times records with a location_id or location_group_id instead of a stop_id. Cases where shape_dist_traveled is provided with a location_id or location_group_id are triggered in the new 🔴 forbidden_shape_dist_traveled error.
• 🟡 stop_without_stop_time: Skips stops that are a part of a location group referenced in stop_times.txt.
• 🟡 stop_too_far_from_shape: Only checks stop times records with a stop_id.

New notices

• 🔴 bidirectional_exit_gate: Triggered when exit gates (pathway_mode=7 in pathways.txt) are bidirectional.
• 🔴 invalid_character: Triggered when text contains invalid characters, such as the replacement character ("�"). Check that text was properly encoded in UTF-8 as required by GTFS.
• 🟡 single_shape_point: Triggered when a shape in shapes.txt only includes one shape point.

Flex new notices

• 🔴 forbidden_shape_dist_traveled: Triggered when shape_dist_traveled is provided with a location_id or location_group_id in stop_times.txt.
• 🔴 forbidden_geography_id: Triggered when a stop_times.txt includes more than one geography id. Only one of stop_id, location_group_id or location_id can be defined for a given entry.

locations.geojson notices

• 🔴 malformed_json: Triggered when locations.geojson is malformed.
• 🔴 invalid_geometry: Triggered when polygon in locations.geojson is unparsable or invalid. Each polygon must be valid by the definition of the OpenGIS Simple Features Specification, section 6.1.11.
• 🔴 missing_required_element: Triggered when an element that's required in the geoJSON file is missing.
• 🔴 unsupported_geo_json_type: Triggered when geoJSON type is a value other than "FeatureCollection".
• 🔴 unsupported_feature_type: Triggered when feature type is a value other than "Feature".
• 🔴 unsupported_geometry_type: Triggered when the geometry type is a value other than "Polygon" or "MultiPolygon".

booking_rules.txt notices

• 🔴 forbidden_prior_day_booking_field_value: Triggered when a forbidden field value is present for a prior-day booking rule.
• 🔴 forbidden_prior_notice_start_day: prior_notice_start_day value is forbidden when prior_notice_duration_max is set.
• 🔴 forbidden_prior_notice_start_time: prior_notice_start_time value is forbidden when prior_notice_start_day value is not set
• 🔴 forbidden_real_time_booking_field_value: A forbidden field value is present for a real-time booking rule.
• 🔴 forbidden_same_day_booking_field_value: A forbidden field value is present for a same-day booking rule.
• 🔴 invalid_prior_notice_duration_min: The prior_notice_duration_max field value needs to be greater or equal to the prior_notice_duration_min field value.
• 🔴 missing_prior_notice_duration_min: prior_notice_duration_min value is required for same day booking_type
• 🔴 missing_prior_day_booking_field_value: prior_notice_last_day and prior_notice_last_time values are required for prior day booking_type.
• 🔴 missing_prior_notice_start_time: prior_notice_start_time value is required when prior_notice_start_day value is set
• 🔴 prior_notice_last_day_after_start_day: prior_notice_last_day should not be greater than the prior_notice_start_day.

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility Database in this spreadsheet.

Other notable improvements

• Bug Fix: Validation report summary shows Unix time 0 for feed_start_date and feed_end_date when feed_info.txt exists with empty date values
• Bug Fix: Count appears as 1 for block even when no blocks exist
• Optimisation: Do not run validators on columns that are not present
• feat: Include service window in Summary report

New Contributors

• @praneethd7 made their first contribution in #1753

PR List

• feat: Changed notice migration generation. Added zip file to release. by @jcpitre in #1736
• feat: documented the changes made in PR 1633 by @qcdyx in #1748
• Feat: 1639 release process for minor versions by @jcpitre in #1657
• feat: notify mobility-feeds-api when the web validator is updated by @cka-y in #1760
• fix: set transfer from and to stop ids as conditional required fields by @davidgamez in #1772
• feat: add performance assessment to acceptance tests by @cka-y in #1771
• feat: New rule - SingleShapePointValidator for flagging shapes with a single shape point by @praneethd7 in #1753
• Docs: Fix typo in FEATURES.md by @isabelle-dr in #1777
• fix: Display empty values for feed_start_date and feed_end_date if there's… by @qcdyx in #1798
• fix: Count appears as 1 for block even when no blocks exist by @qcdyx in #1802
• fix: Added a validator for bad networkId foreign key in fareLegRules by @jcpitre in #1804
• feat: added new field for rule priority in GtfsFareLegRuleSchema by @qcdyx in #1806
• feat: 1791 feed infotxt should be added to missing required file when translationstxt exists by @qcdyx in #1803
• feat: 1786 exit gates pathway mode7 must not be bidirectional by @qcdyx in https://github....

v5.0.1

This minor release was created to fix a key bug that was preventing feed URLs from running successfully in the Web UI. This bug only impacted the Web UI, not the desktop app or CLI tool.

For more details on our 5.0.0 release, you can read about them here.

PR List

• chore: remove credentials on ui build by @davidgamez in #1700
• chore(deps-dev): bump follow-redirects from 1.15.5 to 1.15.6 in /web/client by @dependabot in #1717
• chore(deps-dev): bump vite from 4.3.9 to 4.5.2 in /web/client by @dependabot in #1705
• chore(deps-dev): bump word-wrap from 1.2.3 to 1.2.5 in /web/client by @dependabot in #1697
• docs: Automated update of NOTICE_MIGRATION.md by @github-actions in #1716
• hotfix: jobId incorrectly set in the UI by @cka-y in #1723

Full Changelog: v5.0.0...v5.0.1

v5.0.0

Highlights

Reducing the noisiness of errors

There have been repeated issues with the sensitivity of
trip_distance_exceeds_shape_distanceand equal_shape_distance_diff_coordinates. No more! We did an extensive review of real life GTFS data from the Mobility Database and decided on thresholds for these two errors. Now data producers will only see these errors when they’re a significant issue for data quality. Below the thresholds, these issues will generate a warning instead.

Breaking changes: JSON report improvements

• Issue #1652: Use ISO_ZONED_DATE_TIME for validatedAt field to be machine readable.
• Issue #1685 Change gtfsComponents label to gtfsFeatures.

Notice changes

• equal_shape_distance_diff_coordinates: Error is triggered when there's 1.11 metres or more distance between the different shape points. Discussion here.
• trip_distance_exceeds_shape_distance: Error is triggered when there's 11.1 metres or more distance between the last stop point and the last shape point. Discussion here.
• stop_without_zone_id changed from 🔴 ERROR to ⚪ INFO and updated so it is only triggered when there are route-based fare related fields in fare_rules.txt . Discussed in the spec in google/transit#432.

New notices

• route_networks_specified_in_more_than_one_file: New 🔴 ERROR for Fares v2 data. Added to spec in google/transit#405.
• equal_shape_distance_diff_coordinates_distance_below_threshold: New 🟡 WARNING that is triggered when the distance between the different shape points is less than 1.11 metres.
• trip_distance_exceeds_shape_distance_below_threshold: New 🟡 WARNING that is triggered when the distance between the last stop point and the last shape point is less than 11.1 metres.

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility Database in this spreadsheet.

Other notable improvements

• "Error authorizing upload" bug fix on the Web UI
• Allow the validator to accept a "time of validation" argument in the cli
• Generate new CLI package installers on each merge to master
• Add outstanding GTFS features to the validator summary report. Full list of features here.

New Contributors

• @michaelandrewkearney made their first contribution in #1693

PR List

• docs: Update NOTICE_MIGRATION.md by @emmambd in #1599
• Docs: Add mention to update version number on gtfs.org by @isabelle-dr in #1600
• feat: Support injection of arbitrary types via ValidationContext by @aababilov in #1607
• fix: Add label to step that creates an issue by @fredericsimard in #1610
• fix: cypress error scenario on downloading rules definitions file by @davidgamez in #1617
• feat: Update references to Dataset Publishing Guidelines from #1601 by @emmambd in #1609
• Update release process so we run analytics before release is launched by @emmambd in #1612
• fix: validator version displayed in validator report summary by @jcpitre in #1627
• fix: Correct links to rule documentation by adding -rule by @bradyhunsaker in #1629
• fix: decreasing_or_equal_stop_time_distance includes prevShapeDistTraveled by @cka-y in #1631
• feat: #1619 Added a bit of documentation to refer to gtfs-validator-example. by @jcpitre in #1635
• fix: Feed generates error to authorize upload by @davidgamez in #1633
• refactor: Replace CurrentDateTime with DateForValidation and use LocalDate by @bradyhunsaker in #1636
• docs: Update PULL_REQUEST_TEMPLATE.md by @emmambd in #1622
• fix: authorize upload error on create job endpoint by @davidgamez in #1642
• fix: Corrected a problem where the version was null in the desktop version. by @jcpitre in #1643
• feat: Add an argument for the cli (--date) that specifies the date to use during verification. by @bradyhunsaker in #1628
• fix: Corrected a problem where code was not multithread safe by @jcpitre in #1653
• Feat: 1640 add remaining gtfs features to the validator by @qcdyx in #1656
• feat: removed route name and agency information components by @qcdyx in #1659
• Minor documentation changes by @jcpitre in #1658
• feat: added GtfsNetworkSchema by @qcdyx in #1660
• feat: Added features document. Some corrections to components presence by @jcpitre in #1662
• fix: prevent exception raised when serviceId is missing from calendar file by @davidgamez in #1646
• Fix: replace GitRunners with GitHub's own runners by @fredericsimard in #1664
• Fix: replaced Gitrunners with GitHub's own servers by @fredericsimard in #1673
• fix: reformatted doc using HTML table format by @qcdyx in #1669
• fix: update gradle wrapper by @qcdyx in #1677
• feat: generated CLI package installer by @cka-y in #1680
• feat: filter out acceptance tests from validator usage report by @cka-y in #1683
• chore: add skip version check to the master build by @davidgamez in #1684
• feat: validation rules for networks and route_networks by @cka-y in #1671
• fix: cypress tests pointing to production are triggered on PRs checks by @davidgamez in #1687
• feat: 1651 web validator retries with bad zip file 2 by @qcdyx in #1686
• fix: table not properly rendered in notices by @cka-y in #1695
• feat: used the standard date time formatter by @qcdyx in #1691
• feat: replace GTFS Components language to GTFS features by @cka-y in #1696
• feat: threshold of 1.11m on equal_shape_distance_diff_coordinates by @cka-y in #1675
• fix: Added a execution result file by @jcpitre in #1690
• feat: reformat trip and shape dist validator by @cka-y in #1676
• feat: display ERRORS first on validator documentation by @cka-y in #1667
• fix: make stop_without_zone_id conditional on fare rule type (#1663) by @michaelandrewkearney in #1693
• fix: correct field name in missing_recommended_field_notice (#1574) by @michaelandrewkearney in #1706
• fix: 1707 web validator waits for validation forever by @jcpitre in #1711
• docs: update deprecated architecture information by @michaelandrewkearney in #1694
• fix: setting package installer java version to 17.0 by @cka-y in #1709
• feat: 1689 web validator app consumes execution results json by @davidgamez in #1712
• feat: Add examples table for mixed_case_recommended_field by @emmambd in #1708

Full Changelog: v4.2.0...v5.0.0

v4.2.0

Overview

This release includes

• Support for time variable fares that was adopted in GTFS in July 2023
• A new summary section of the validation report, including tags to see if a feed has GTFS components like Blocks, Frequencies, or Fares v2
• Automatic generation of the RULES documentation on https://gtfs-validator.mobilitydata.org/rules.html
• New releases published to Maven

New notices

• invalid_input_files_in_subfolder 🔴 ERROR, added to the spec in google/transit#379
• trip_distance_exceeds_shape_distance 🔴 ERROR, added to the spec in google/transit#380
• missing_bike_allowance 🟠WARNING, added to best practices in MobilityData/GTFS_Schedule_Best-Practices#55
• missing_feed_contact_email_and_url 🟠WARNING
• unused_parent_station ⚪ INFO

Notices for Fares v2 - time variable fares, added to the spec in google/transit#357

• timeframe_overlap 🔴 ERROR
• timeframe_only_start_or_end_time_specified 🔴 ERROR
• timeframe_start_or_end_time_greater_than_twenty_four_hours 🔴 ERROR

Changed notices

• Downgraded platform_without_parent_station 🟠WARNING → ⚪ INFO
• Replaced missing_timepoint_column 🟠WARNING → missing_recommended_column 🟠WARNING

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility database in this spreadsheet.

Full Changelog: v4.1.0...v4.2.0

New Contributors

• @mijogu made their first contribution in #1396
• @ryon made their first contribution in #1397
• @jcpitre made their first contribution in #1439
• @lottspot made their first contribution in #1453
• @cka-y made their first contribution in #1469
• @qcdyx made their first contribution in #1492
• @bradyhunsaker made their first contribution in #1493
• @Whoops made their first contribution in #1519
• @emmambd made their first contribution in #1550
• @1Maxnet1 made their first contribution in #1546
• @fredericsimard made their first contribution in #1558

What's Changed

Rules

• fix: Downgraded platform_without_parent_station notice to info by @jcpitre in #1444
• feat: Add a rule that stations (location_type 1) must be the parent_station of some stop (location_type 0). by @bradyhunsaker in #1493
• feat: Added missing_recommended_column notice by @jcpitre in #1470
• feat: Validation for timeframes.txt by @bdferris-v2 in #1518
• feat: 1423 only warn if feed contact email or url are both unset by @qcdyx in #1514
• feat: validator for ferry trips + bikes allowance by @cka-y in #1510
• feat: 1504 add new error if a gtfs dataset has gtfs files in sub folders by @qcdyx in #1535
• feat: trip vs shape distance validation by @cka-y in #1553
• fix: comments on #1587 by @cka-y in #1592

Bug fixes

• fix: Fix typo in TimeframeOverlapNotice class name by @bdferris-v2 in #1569
• Fix: typo in config.yml by @isabelle-dr in #1435
• feat: 1532 support mac created zip files by @qcdyx in #1543
• fix: Mixed Case false positives by @briandonahue in #1430
• fix: don't throw exception on calendar with no days by @Whoops in #1519
• fix: Update the "Help" menu with current command line parameters#1116 by @cka-y in #1502
• fix: Package installers CI step failing by @davidgamez in #1568
• Fix for deprecated of set-output in GH actions by @fredericsimard in #1571
• bug: initialized tooltip in the report.html by @qcdyx in #1581
• fix: Remove triage action & add exclude paths by @fredericsimard in #1576
• fix: fix OutOfBoundsException when agency.txt is missing in the input data… by @qcdyx in #1551
• Fix: Restore missing CSS by @briandonahue in #1457
• fix: Modified to accept mime type encountered on Windows by @jcpitre in #1439
• fix: Web Validator hangs after supplying non-zip URL #1445 by @cka-y in #1481
• fix: Web validator allows non-zip files to be uploaded #1440 by @cka-y in #1482

New summary section of report

• feat: Add metadata to report by @briandonahue in #1389
• fix: Bug in HTML report generation where feed metadata can be null by @bdferris-v2 in #1451
• feat: Added route names component by @cka-y in #1469
• Issue/1463 by @cka-y in #1471
• feat: 1465 add a route colors gtfs component by @qcdyx in #1492
• feat: 1466 add a transfers gtfs component by @qcdyx in #1501
• feat: Provide an explanation to users about what a GTFS Component is #1505 by @cka-y in #1512
• feat: added rest of GTFS components by @cka-y in #1516
• feat: Add overview section to the Json report by @jcpitre in #1531

Generate documentation automatically

• fix: Refactor MainTest to avoid stubbing System.exit() by @bdferris-v2 in #1410
• feat: Add @GtfsValidationNotice annotations to all notice classes by @bdferris-v2 in #1366
• feat: Add unit-test check for @GtfsValidationNotice annotation and notices that I missed by @bdferris-v2 in #1413
• feat: Include section references in the notice output schema. by @bdferris-v2 in #1416
• feat: Add Notice documentation header check. by @bdferris-v2 in #1422
• feat: Change Notice field comment format and unit-test to enforce they exist by @bdferris-v2 in #1421
• Fix: Switch javadoc aggregation plugin to non-jar version. by @bdferris-v2 in #1432
• feat: Add a few missing "Affected files" references by @bdferris-v2 in #1427
• fix: Use correct import for @nullable annotation. by @bdferris-v2 in #1438
• Feat: Bulk update Notice Javadoc to include content from RULES.md. by @bdferris-v2 in #1428
• feat: Add documentation update + unit-test for new notice doc conventions. by @bdferris-v2 in #1429
• feat: Split the notice doc comment into a short summary string and additional documentation. by @bdferris-v2 in #1452
• feat: Additional test to enforce documentation comment conventions by @bdferris-v2 in #1450
• Fix: bug in Notice documentation comment splitting. by @bdferris-v2 in #1479
• feat: Resolve notice severity level from @GtfsValidationNotice annotation. by @bdferris-v2 in #1475
• Feat: Remove SeverityLevel from ValidationNotice constructor by @bdferris-v2 in #1489
• feat: Remove unit-test that RULES.md is up-to-date with Notice classes. by @bdferris-v2 in #1517
• Remove unit-test that RULES.md is up-to-date with Notice classes by @qcdyx in #1522

Web-based validator improvements

• feat: integrate Cypress for E2E testing by @mijogu in #1396
• chore: ignore web packages on acceptance and e2e tests by @davidgamez in #1403
• feat: report link sharing by @...

v4.1.0

Overview

This release includes the addition of the Web version available at https://gtfs-validator.mobilitydata.org/, new notices, support for Fare Media that was adopted in GTFS in March 2023, and initial support to programmatically generate the notice documentation.
See the notice mapping between releases in NOTICE_MIGRATION.md.

New notices

• invalid_currency_amount 🔴 ERROR
• missing_stop_name 🔴 ERROR
• stop_without_location 🔴 ERROR
• too_many_rows 🔴 ERROR
• duplicate_fare_media 🟠 WARNING
• expired_calendar 🟠 WARNING
• missing_feed_info_date 🟠 WARNING
• mixed_case_recommended_field 🟠 WARNING
• route_long_name_contains_short_name 🟠 WARNING
• trip_coverage_not_active_for_next7_days 🟠 WARNING

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility database in this spreadsheet.

What's Changed

Web-based validator

• feat: Web Based Validator by @KClough in #1317
• feat: Web Validator Client Display Errors by @KClough in #1383
• feat: Add Sentry Error Monitoring to web validator client + service by @KClough in #1384
• feat: add newrelic agent to web service by @KClough in #1387
• chore: Web validator ide warnings by @KClough in #1388
• chore: add web validator information to README by @davidgamez in #1399

Bug fixes

• bug: upgrade GitHub Workflow actions to Node16 compatible versions by @bdferris-v2 in #1271
• fix: Don't ignore invalid language codes in RowParser.asLanguageCode by @asvechnikov2 in #1291
• Fix miscellaneous error-prone findings by @aababilov in #1312
• fix: byTranslationKey() for multi-key schemas without translations.txt support by @bdferris-v2 in #1302
• fix: broken link in RULES.md by @isabelle-dr in #1332
• Fix: Fix bug triage workflow by @isabelle-dr in #1336
• Fix: whitespace and anchor tag by @briandonahue in #1342
• fix: Failing Docker Image workflow by @davidgamez in #1386
• fix: Run all sub-project tests in test_pack_doc.yml workflow by @bdferris-v2 in #1382
• fix: Bump validation workflows to use large machine instances by @bdferris-v2 in #1320

Rules

• fix: Drop DuplicateFareRuleZoneIdFieldsValidator and replace with multi-column @PrimaryKey for fare_rules.txt by @bdferris-v2 in #1297
• feat: Date validity - Calendar Expiration + Trip Coverage by @KClough in #1289
• Fix 1198: separate required column from required value by @briandonahue in #1344
• fix: Add notice for missing stop name for specific location types by @briandonahue in #1316
• fix: add field value to mixed case notice by @briandonahue in #1393
• fix: generate errors when missing lat/long for stops, stations, entrances by @briandonahue in #1321
• fix 878: require agency_id when more than one agency, warn when only one by @briandonahue in #1318
• feat 267: route.long_name should not contain route.short_name by @briandonahue in #1325
• feat: Add MixedCase validation (#881) by @briandonahue in #1340
• feat: Add code for computing service coverage windows and trip counts by @bdferris-v2 in #1351

Documentation updates

• Docs: documentation updates by @isabelle-dr in #1288
• Docs: Fix RULES.md notice name for 2 notices by @isabelle-dr in #1323
• Doc: RULES.md update by @isabelle-dr in #1307
• Docs: Update RULES.md by @isabelle-dr in #1359
• Docs: Update ACCEPTANCE_TESTS.md by @isabelle-dr in #1385
• Docs: Update notice migration file by @isabelle-dr in #1357
• fix: update RULES.md with missing rule by @briandonahue in #1330

Generate the doc programmatically

• feat: Initial entry for @GtfsValidationNotice annotation by @bdferris-v2 in #1361
• feat: Update Notice source code such that field name comments match documentation in RULES.md by @bdferris-v2 in #1346
• feat: Add unit-test to check consistency of Notice field names. by @bdferris-v2 in #1345
• fix: Inconsistencies in RULES.md against underlying source by @bdferris-v2 in #1341
• feat: Add unit test to verify that every Notice has an entry in RULES.md by @bdferris-v2 in #1331
• feat: Extract notice docs from source-code for in-app use and schema export by @bdferris-v2 in #1365

Acceptance tests improvements

• feat: Update acceptance test report to include changes in WARNINGS. by @bdferris-v2 in #1354
• feat: Fail acceptance tests on significant change in warnings by @bdferris-v2 in #1377

General maintenance + other good stuff

• Use smaller integer field types for GTFS entity classes by @aababilov in #1273
• Introduce GtfsEntityBuilder interface by @aababilov in #1286
• Use int instead of long to keep CSV row number by @aababilov in #1287
• Generate registries of tables and validators by @aababilov in #1290
• Avoid unchecked conversion in table and validator registries by @aababilov in #1293
• build(deps): bump certifi from 2021.5.30 to 2022.12.7 in /scripts/mobility-database-harvester by @dependabot in #1299
• Separate table loading logic from table structure by @aababilov in #1284
• Do not invoke single file validators if table indices were not generated by @aababilov in #1300
• Always invoke single file validators (revert #1300) by @aababilov in #1303
• Discover tables and validators in runtime using io.github.classgraph by @aababilov in #1311
• Initialize one field per line in the generated code by @aababilov in #1314
• feat: Refactor JSON report comparison for better diffing in unit-tests. by @bdferris-v2 in #1352
• feat: Move key annotation and entity interfaces into a new model module by @bdferris-v2 in #1362
• feat: Initial support for GTFS Fares v2 - fare media by @bdferris-v2 in #1305
• feat: Add a more descriptive test failure message to NoticeFieldsTest. by @bdferris-v2 in #1369
• chore: Add unit tests to AnyTableLoader by @davidgamez in #1380

New Contributors

• @asvechnikov2 made their first contribution in #1291
• @briandonahue made their first contribution in #1321
• @davidgamez made their first contribution in #1386

Full Changelog: v4.0.0...v4.1.0

v4.0.0

Overview

This is tagged as a major release because adding the rules for Fares v2 base implementation is affecting the validity of existing datasets. We started to evaluate the impact of new rules on production data in 2022 with the acceptance tests, which runs this validator on all datasets from the Mobility Database, and reports if additional errors appear. When new errors appear, we publish a major release.

New notices

GTFS-Fares v2 base implementation, added in google/transit/pull/286

• fare_transfer_rule_duration_limit_type_without_duration_limit 🔴 ERROR
• fare_transfer_rule_duration_limit_without_type 🔴 ERROR
• fare_transfer_rule_invalid_transfer_count 🔴 ERROR
• fare_transfer_rule_missing_transfer_count 🔴 ERROR
• fare_transfer_rule_with_forbidden_transfer_count 🔴 ERROR
• invalid_currency_amount 🔴 ERROR

Transfers, changes in google/transit/pull/303 and google/transit/pull/284

• transfer_with_invalid_stop_location_type 🔴 ERROR
• transfer_with_invalid_trip_and_route 🔴 ERROR
• transfer_with_invalid_trip_and_stop 🔴 ERROR
• transfer_with_suspicious_mid_trip_in_seat 🟠 WARNING

Best Practice rules

• missing_recommended_file (applies to feed_info.txt) 🟠 WARNING
• missing_recommended_field (applies to feed_info.txt start_date, end_date, email, URL, version) 🟠 WARNING)
• feed_expiration_date_7_days (replacing feed_expiration_date) 🟠 WARNING
• feed_expiration_date_30_days (replacing feed_expiration_date) 🟠 WARNING

Impacts on existing data

On all datasets MobilityData ran the analytics on, we found that 73 sources would be affected by this release. The datasets affected mostly contain earlier versions of Fares v2 that don't match what was adopted in the spec, and uncertain cases, future versions.
The detailed analysis is in the Fares v2 base implementation for the validator document.

Producers affected

• Trillium Transit
• San Diego Metropolitan Transit System

Complete list

See the complete list of datasets affected 1/2

URL	Notices
http://data.trilliumtransit.com/gtfs/nts-ca-us/nts-ca-us.zip	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/victorville-ca-us/victorville-ca-us.zip	duplicate_key,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/tracy-ca-us/tracy-ca-us.zip	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/downey-ca-us/downey-ca-us.zip	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/weaverville-ca-us/weaverville-ca-us.zip	duplicate_key,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/laketransit-ca-us/laketransit-ca-us.zip	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/auburntransit-ca-us/auburntransit-ca-us.zip	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/cityofridgecrest-ca-us/cityofridgecrest-ca-us.zip	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://www.sdmts.com/google_transit_files/google_transit.zip	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/lagunabeach-ca-us/lagunabeach-ca-us.zip	invalid_currency_amount
http://data.trilliumtransit.com/gtfs/goldcountrystage-ca-us/goldcountrystage-ca-us.zip	duplicate_key,fare_transfer_rule_with_forbidden_transfer_count,foreign_key_violation,invalid_currency_amount
https://transitfeeds.com/p/santa-cruz-metro/343/latest/download	duplicate_key,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/redding-ca-us/redding-ca-us.zip	duplicate_key,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/corona-ca-us/corona-ca-us.zip	fare_transfer_rule_missing_transfer_count,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/gtrans-ca-us/gtrans-ca-us.zip	fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/vctc-ca-us/vctc-ca-us.zip	fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/sanbenitocounty-ca-us/sanbenitocounty-ca-us.zip	fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/delnorte-ca-us/delnorte-ca-us.zip	fare_transfer_rule_with_forbidden_transfer_count,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/plumas-ca-us/plumas-ca-us.zip	foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/sagestage-ca-us/sagestage-ca-us.zip	foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/bigbear-ca-us/bigbear-ca-us.zip	foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/mercedthebus-ca-us/mercedthebus-ca-us.zip	foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/thousandoaks-ca-us/thousandoaks-ca-us.zip	foreign_key_violation,invalid_currency_amount

See the complete list of datasets affected 2/2

URL	Notices
http://data.trilliumtransit.com/gtfs/lassen-ca-us/lassen-ca-us.zip	foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/banning-ca-us/banning-ca-us.zip	foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/morongobasin-ca-us/morongobasin-ca-us.zip	foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/amador-ca-us/amador-ca-us.zip	foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/glenn-ca-us/glenn-ca-us.zip	foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/...

v3.1.1

Bug fixes

• fix: Bug with ssl network connections + Java module permissions. by @bdferris-v2 in #1182
• fix: Temporary fix for MacOS validator app corruption by @bdferris-v2 in #1186

Enhancements

• feat: recommended annotation by @KClough in #1149
• chore: add bin folder to .gitignore by @KClough in #1196

New Contributors

• @KClough made their first contribution in #1149 🎉

Full Changelog: v3.1.0...v3.1.1

v3.1.0

📣 New feature alert! Users can now run the validator as a packaged app and see the results in the browser. This work is described in the epic issue #1124

Enhancements

• feat: Packaged Executable by @bdferris-v2 in #1125
• feat: Implements the HTML output MVP by @maximearmstrong in #1127
• docs: Update RULES.md to include NOTICES.md information & delete NOTICES.md by @isabelle-dr in #1132
• feat: Initial support for validation app GUI by @bdferris-v2 in #1146
• fix: Replace Java wildcard imports with explicit imports. by @bdferris-v2 in #1161
• feat: Refactor handling of project version numbers and tags by @bdferris-v2 in #1147
• fix: Update acceptance_test.yml to reflect new -cli.jar naming. by @bdferris-v2 in #1168
• test: Add path parsing tests by @barbeau in #1155
• feat: Initial entry for a workflow to package the gui application. by @bdferris-v2 in #1166
• feat: Persist GUI app settings between executions by @bdferris-v2 in #1169
• feat: Check for new version of app during validation by @bdferris-v2 in #1176
• feat: Add an icon for the GUI application by @bdferris-v2 in #1179

Bugs fixed

• fix: Docker Image push to GitHub Container Registry by @maximearmstrong in #1142
• ci: Remove upload of validation reports to Google Cloud by @lionel-nj in #1102

Documentation improvements

• docs: Update instructions to run the validator by @isabelle-dr in #1115
• docs: Add gradle commands shadowJar and aggregateJavadocs by @barbeau in #1122
• docs: Remove dollar sign from commands by @barbeau in #1138
• Docs: fix typo for Missing RequiredFileNotice by @isabelle-dr in #1130
• Update README.md to reference the GUI application. by @bdferris-v2 in #1173
• fix: Fixes titles in RULES.md by @maximearmstrong in #1177

Maintenance and upkeep improvements

• fix: Set encoding to UTF-8 for Javadoc generation by @barbeau in #1121
• fix: SystemStubs test dependency by @maximearmstrong in #1140
• fix: Modify end_to_end.yml to be more specific about which gtfs-validator jar is used by @bdferris-v2 in #1131
• ci: Change Java distribution adopt->temurin by @barbeau in #1118
• ci: include multiple versions of java and OS in test_pack_dock workflow by @lionel-nj in #1093
• feat: Adds .nosync to .gitignore by @maximearmstrong in #1136
• test: SystemStubs instead of SystemLambda by @ed-g in #1092
• ci: Remove ubuntu-20.04 from CI by @barbeau in #1141
• feat: Refactor cli.Main into ValidationRunner for easier reuse. by @bdferris-v2 in #1145

Contributors to this release

• @barbeau always a pleasure 🤝
• 👋 welcome @bdferris-v2
• 👋 welcome @ed-g
• @isabelle-dr
• @maximearmstrong

Full Changelog: v3.0.1...v3.1.0