A Scalable Reinforcement Learning Algorithm for Scheduling Railway Lines
This paper describes an algorithm for scheduling bidirectional railway lines (both single- and multi-track) using a reinforcement learning (RL) approach. The goal is to define the track allocations and arrival/departure times for all trains on the line, given their initial positions, priority, halt...
Saved in:
| Published in: | IEEE transactions on intelligent transportation systems Vol. 20; no. 2; pp. 727 - 736 |
|---|---|
| Main Author: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
New York
IEEE
01.02.2019
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects: | |
| ISSN: | 1524-9050, 1558-0016 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | This paper describes an algorithm for scheduling bidirectional railway lines (both single- and multi-track) using a reinforcement learning (RL) approach. The goal is to define the track allocations and arrival/departure times for all trains on the line, given their initial positions, priority, halt times, and traversal times, while minimizing the total priority-weighted delay. The primary advantage of the proposed algorithm compared to exact approaches is its scalability, and compared to heuristic approaches is its solution quality. Efficient scaling is ensured by decoupling the size of the state-action space from the size of the problem instance. Improved solution quality is obtained because of the inherent adaptability of reinforcement learning to specific problem instances. An additional advantage is that the learning from one instance can be transferred with minimal re-learning to another instance with different infrastructure resources and traffic mix. It is shown that the solution quality of the RL algorithm exceeds that of two prior heuristic-based approaches while having comparable computation times. Two lines from the Indian rail network are used for demonstrating the applicability of the proposed algorithm in the real world. |
|---|---|
| AbstractList | This paper describes an algorithm for scheduling bidirectional railway lines (both single- and multi-track) using a reinforcement learning (RL) approach. The goal is to define the track allocations and arrival/departure times for all trains on the line, given their initial positions, priority, halt times, and traversal times, while minimizing the total priority-weighted delay. The primary advantage of the proposed algorithm compared to exact approaches is its scalability, and compared to heuristic approaches is its solution quality. Efficient scaling is ensured by decoupling the size of the state-action space from the size of the problem instance. Improved solution quality is obtained because of the inherent adaptability of reinforcement learning to specific problem instances. An additional advantage is that the learning from one instance can be transferred with minimal re-learning to another instance with different infrastructure resources and traffic mix. It is shown that the solution quality of the RL algorithm exceeds that of two prior heuristic-based approaches while having comparable computation times. Two lines from the Indian rail network are used for demonstrating the applicability of the proposed algorithm in the real world. |
| Author | Khadilkar, Harshad |
| Author_xml | – sequence: 1 givenname: Harshad orcidid: 0000-0003-3601-778X surname: Khadilkar fullname: Khadilkar, Harshad email: harshad.khadilkar@tcs.com organization: TCS Research, Tata Consultancy Services, Mumbai, India |
| BookMark | eNp9kEFrwkAQhZdioWr7A0ovgZ5jd3azSfYo0lZBKKg9L5vdia7Ejd1Eiv--CUoPPfQ0w8z75jFvRAa-9kjII9AJAJUvm8VmPWEU8gnLmYRU3JAhCJHHlEI66HuWxJIKekdGTbPvpokAGJL5NFobXemiwmiFzpd1MHhA30ZL1ME7v42m1bYOrt0dom7ZqXdoT1W_WGlXfetztHQem3tyW-qqwYdrHZPPt9fNbB4vP94Xs-kyNkzyNhbUFLagAEmGmWClTahOmYDSZDwvNEcLXOYFUmZ4ai0IK60tOU8oLUsjCj4mz5e7x1B_nbBp1b4-Bd9ZKgaZ4HlGhexUcFGZUDdNwFIdgzvocFZAVR-Y6gNTfWDqGljHZH8Y41rdutq3ofv0X_LpQjpE_HXKucgklfwHqmB6Wg |
| CODEN | ITISFG |
| CitedBy_id | crossref_primary_10_1007_s10479_021_04178_x crossref_primary_10_1016_j_jrtpm_2024_100483 crossref_primary_10_1177_03611981241302335 crossref_primary_10_1016_j_ifacol_2023_10_1503 crossref_primary_10_1016_j_ifacol_2024_07_358 crossref_primary_10_1515_comp_2024_0014 crossref_primary_10_1109_TITS_2021_3131202 crossref_primary_10_1109_JIOT_2022_3152381 crossref_primary_10_1109_TNNLS_2024_3357494 crossref_primary_10_1007_s11227_025_07399_0 crossref_primary_10_1016_j_tre_2024_103900 crossref_primary_10_1016_j_swevo_2024_101517 crossref_primary_10_1109_MITS_2020_2970180 crossref_primary_10_1007_s00521_021_06129_w crossref_primary_10_1109_TITS_2025_3547473 crossref_primary_10_1016_j_cie_2025_111329 crossref_primary_10_3390_app10238367 crossref_primary_10_1080_21680566_2023_2179461 crossref_primary_10_1016_j_jrtpm_2023_100394 crossref_primary_10_1109_TCYB_2023_3312647 crossref_primary_10_1109_TETC_2020_3030984 crossref_primary_10_1177_03611981251339168 crossref_primary_10_1109_TIV_2023_3305536 crossref_primary_10_1109_TITS_2023_3344468 crossref_primary_10_3390_en14164776 crossref_primary_10_3390_app13179547 crossref_primary_10_1016_j_tre_2020_102176 crossref_primary_10_1007_s13177_025_00500_0 crossref_primary_10_1016_j_ress_2023_109136 crossref_primary_10_1109_JAS_2023_123501 crossref_primary_10_3390_rs13122377 crossref_primary_10_1109_TITS_2023_3319135 crossref_primary_10_1016_j_trc_2022_103679 crossref_primary_10_1109_TNNLS_2023_3309689 crossref_primary_10_1177_0361198120917388 crossref_primary_10_1111_exsy_13315 crossref_primary_10_1109_TITS_2023_3305074 crossref_primary_10_1016_j_jmsy_2022_11_010 crossref_primary_10_3390_app14198642 crossref_primary_10_1016_j_jrtpm_2023_100427 crossref_primary_10_1016_j_trb_2023_02_015 crossref_primary_10_1061_JMENEA_MEENG_4883 crossref_primary_10_1016_j_trc_2025_105215 crossref_primary_10_1016_j_trpro_2021_07_119 crossref_primary_10_1038_s41598_023_29526_8 crossref_primary_10_1007_s12652_020_02862_7 crossref_primary_10_1109_TITS_2022_3229527 crossref_primary_10_3390_rs16234436 crossref_primary_10_1109_TITS_2021_3131637 crossref_primary_10_1016_j_tre_2022_102712 crossref_primary_10_1016_j_asoc_2023_110590 crossref_primary_10_1016_j_eswa_2021_114663 crossref_primary_10_1093_tse_tdad007 crossref_primary_10_1109_TCSS_2021_3119944 crossref_primary_10_1007_s10845_021_01847_3 crossref_primary_10_1016_j_trc_2023_104155 crossref_primary_10_1016_j_trc_2024_104970 crossref_primary_10_1016_j_trc_2024_104976 crossref_primary_10_1109_TASE_2023_3338695 crossref_primary_10_1016_j_trc_2023_104237 crossref_primary_10_1080_00207543_2025_2481184 crossref_primary_10_1109_TSMC_2023_3346401 crossref_primary_10_3390_rs17121972 crossref_primary_10_3390_app14041552 |
| Cites_doi | 10.1287/opre.14.5.801 10.1016/j.cor.2008.12.012 10.1016/j.trb.2015.03.004 10.1109/TITS.2012.2234118 10.1016/S0377-2217(77)81007-2 10.1016/j.trb.2013.10.013 10.1016/0305-0548(94)90099-X 10.1109/TITS.2014.2320757 10.1109/TITS.2014.2379617 10.1023/A:1021255214371 10.1016/j.jrtpm.2014.04.001 10.1016/S0022-0000(75)80008-0 10.1016/S0377-2217(97)00317-2 10.1080/07408179808966488 10.1177/0954409714565654 10.1049/pe:19950408 10.1016/j.trc.2012.11.001 10.1016/j.trb.2016.01.004 10.1016/0191-2615(95)00022-4 10.1016/j.jrtpm.2016.06.004 10.23919/ACC.2017.7963829 10.1109/TITS.2015.2446985 10.1016/j.neunet.2014.09.003 10.1023/A:1009672832658 10.1016/j.ejor.2006.10.034 10.1287/trsc.8.1.65 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD FR3 JQ2 KR7 L7M L~C L~D |
| DOI | 10.1109/TITS.2018.2829165 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Civil Engineering Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Civil Engineering Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Civil Engineering Abstracts |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-0016 |
| EndPage | 736 |
| ExternalDocumentID | 10_1109_TITS_2018_2829165 8357909 |
| Genre | orig-research |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AIBXA AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P PQQKQ RIA RIE RNS ZY4 AAYXX CITATION 7SC 7SP 8FD FR3 JQ2 KR7 L7M L~C L~D |
| ID | FETCH-LOGICAL-c293t-50cbdb01147e752fd40a6251fc738ba3ed1398be02c36dd15d9ddf33400ffc5b3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 88 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000460756900027&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1524-9050 |
| IngestDate | Sun Nov 30 04:27:10 EST 2025 Sat Nov 29 06:34:52 EST 2025 Tue Nov 18 21:52:55 EST 2025 Wed Aug 27 03:06:24 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 2 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c293t-50cbdb01147e752fd40a6251fc738ba3ed1398be02c36dd15d9ddf33400ffc5b3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-3601-778X |
| PQID | 2175387059 |
| PQPubID | 75735 |
| PageCount | 10 |
| ParticipantIDs | crossref_primary_10_1109_TITS_2018_2829165 proquest_journals_2175387059 crossref_citationtrail_10_1109_TITS_2018_2829165 ieee_primary_8357909 |
| PublicationCentury | 2000 |
| PublicationDate | 2019-02-01 |
| PublicationDateYYYYMMDD | 2019-02-01 |
| PublicationDate_xml | – month: 02 year: 2019 text: 2019-02-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on intelligent transportation systems |
| PublicationTitleAbbrev | TITS |
| PublicationYear | 2019 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref35 ref13 ref34 ref12 ref15 ref14 randløv (ref33) 1998 ref11 ref10 sutton (ref7) 2012 ref2 ref1 hirashima (ref25) 2011; 38 ref16 ref19 ref18 (ref36) 2013 zhang (ref28) 1996; 8 ref23 ref26 ref20 hirashima (ref24) 2010 ref22 ref21 grzybowski (ref30) 2011; 19 hill (ref8) 1995; 9 shalev-shwartz (ref32) 2016 ref29 (ref37) 2015 ref9 ref4 zhang (ref27) 1995 ref3 ref6 cai (ref17) 1998; 30 ref5 boyan (ref31) 1992 |
| References_xml | – start-page: 1 year: 2010 ident: ref24 article-title: A reinforcement learning system for transfer scheduling of freight cars in a train publication-title: Proc Int Multiconf Eng Comput Sci – ident: ref14 doi: 10.1287/opre.14.5.801 – start-page: 1 year: 1995 ident: ref27 article-title: A reinforcement learning approach to job-shop scheduling publication-title: Proc Int Joint Conf Artif Intell – ident: ref3 doi: 10.1016/j.cor.2008.12.012 – ident: ref16 doi: 10.1016/j.trb.2015.03.004 – ident: ref22 doi: 10.1109/TITS.2012.2234118 – volume: 19 start-page: 147 year: 2011 ident: ref30 article-title: Monte Carlo analysis of risk measures for blackjack type optimal stopping problems publication-title: Eng Lett – ident: ref34 doi: 10.1016/S0377-2217(77)81007-2 – ident: ref15 doi: 10.1016/j.trb.2013.10.013 – ident: ref4 doi: 10.1016/0305-0548(94)90099-X – ident: ref23 doi: 10.1109/TITS.2014.2320757 – ident: ref11 doi: 10.1109/TITS.2014.2379617 – ident: ref18 doi: 10.1023/A:1021255214371 – year: 2016 ident: ref32 publication-title: Safe Multi-Agent Reinforcement Learning for Autonomous Driving – ident: ref9 doi: 10.1016/j.jrtpm.2014.04.001 – ident: ref1 doi: 10.1016/S0022-0000(75)80008-0 – volume: 8 start-page: 1024 year: 1996 ident: ref28 article-title: High-performance job-shop scheduling with a time-delay TD( $\lambda $ ) network publication-title: Proc Adv Neural Inf Process Syst – ident: ref35 doi: 10.1016/S0377-2217(97)00317-2 – year: 2015 ident: ref37 publication-title: Train timetable and operational data archives – year: 2012 ident: ref7 publication-title: Reinforcement Learning An Introduction – volume: 30 start-page: 481 year: 1998 ident: ref17 article-title: Greedy heuristics for rapid scheduling of trains on a single track publication-title: IIE Trans doi: 10.1080/07408179808966488 – ident: ref26 doi: 10.1177/0954409714565654 – volume: 9 start-page: 201 year: 1995 ident: ref8 article-title: electric railway traction. iv. signalling and interlockings publication-title: Power Engineering Journal doi: 10.1049/pe:19950408 – ident: ref21 doi: 10.1016/j.trc.2012.11.001 – ident: ref29 doi: 10.1016/j.trb.2016.01.004 – year: 1992 ident: ref31 article-title: Modular neural networks for learning context-dependent game strategies – volume: 38 start-page: 242 year: 2011 ident: ref25 article-title: A reinforcement learning method for train marshaling based on movements of locomotive publication-title: IAENG Int J Comput Sci – year: 2013 ident: ref36 publication-title: R A Language and Environment for Statistical Computing – ident: ref5 doi: 10.1016/0191-2615(95)00022-4 – ident: ref19 doi: 10.1016/j.jrtpm.2016.06.004 – ident: ref20 doi: 10.23919/ACC.2017.7963829 – start-page: 463 year: 1998 ident: ref33 article-title: Learning to drive a bicycle using reinforcement learning and shaping publication-title: Proc Int Conf Mach Learn – ident: ref10 doi: 10.1109/TITS.2015.2446985 – ident: ref12 doi: 10.1016/j.neunet.2014.09.003 – ident: ref6 doi: 10.1023/A:1009672832658 – ident: ref2 doi: 10.1016/j.ejor.2006.10.034 – ident: ref13 doi: 10.1287/trsc.8.1.65 |
| SSID | ssj0014511 |
| Score | 2.5585291 |
| Snippet | This paper describes an algorithm for scheduling bidirectional railway lines (both single- and multi-track) using a reinforcement learning (RL) approach. The... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 727 |
| SubjectTerms | Algorithms Allocations Decoupling Delays Heuristic methods Learning (artificial intelligence) Machine learning Processor scheduling Production scheduling Rail transportation Schedules Scheduling Tracking Trains |
| Title | A Scalable Reinforcement Learning Algorithm for Scheduling Railway Lines |
| URI | https://ieeexplore.ieee.org/document/8357909 https://www.proquest.com/docview/2175387059 |
| Volume | 20 |
| WOSCitedRecordID | wos000460756900027&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-0016 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014511 issn: 1524-9050 databaseCode: RIE dateStart: 20000101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEF5q8aAHX1WsVtmDJzHt5rHZ7LGIpUIp0lbpLST7qIXaSpsq_nt3NmkoKIKXEMhMCDPZnfe3CN3IKAwkp8rh5uoEjCYO11o7iUtSPwqkp2xO96XH-v1oPOZPFXRXzsIopWzzmWrCra3ly4VYQ6qsZbwFxmFab4exMJ_VKisGgLNlsVG9wOGEbiqYLuGt0eNoCE1cURPKhi7YkS0bZA9V-bETW_PSOfzfhx2hg8KNxO1c78eoouYnaH8LXLCGum08NAqA0Sg8UBYgVdhcIC4wVSe4PZssltPs9Q2bh4b61dgdGE_Hg2Q6-0y-cA-a4k_Rc-dhdN91inMTHGGMd-ZQIlKZQqTDFKOelgFJTJjjasH8KE18JY3bF6WKeMIPpXSp5FJq3zfLWWtBU_8MVeeLuTpH2DMOGSPaE6FSQSjSJBCGz3h5GhBto7SOyEaSsShAxeFsi1lsgwvCYxB-DMKPC-HX0W3J8p4javxFXANpl4SFoOuosVFXXKy5VewB6KjZfii_-J3rEu2Zd_O857qBqtlyra7QrvjIpqvltf2dvgHYncdq |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB5EBfXgW6xW3YMnMe0m2W2yxyJKi7WIVvEWkn20hdpKmyr-e3e2aREUwUsIZIaEmezO-1uAcxXXmBJce8JePRbx1BPGGC_1aRbGTAXa5XSfW1G7Hb-8iPsluFzMwmitXfOZruCtq-WrkZxiqqxqvYVI4LTeCmcsoLNprUXNAJG2HDpqwDxB-byG6VNR7TQ7j9jGFVewcOijJflmhdyxKj_2Ymdgbrb-92nbsFk4kqQ-0_wOLOnhLmx8gxfcg0adPFoV4HAUedAOIlW6bCApUFW7pD7ojsb9vPdK7ENL3bOWBwfUyUPaH3ykn6SFbfH78HRz3blqeMXJCZ605jv3OJWZyjDWiXTEA6MYTW2g4xsZhXGWhlpZxy_ONA1kWFPK50ooZcLQLmhjJM_CA1gejob6EEhgXbKImkDWtGY1maVMWj7r5xnEtI2zEtC5JBNZwIrj6RaDxIUXVCQo_ASFnxTCL8HFguVthqnxF_EeSntBWAi6BOW5upJi1U2SAGFH7QbExdHvXGew1ujctZJWs317DOv2PWLWgV2G5Xw81SewKt_z_mR86n6tL2NbyrE |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Scalable+Reinforcement+Learning+Algorithm+for+Scheduling+Railway+Lines&rft.jtitle=IEEE+transactions+on+intelligent+transportation+systems&rft.au=Khadilkar%2C+Harshad&rft.date=2019-02-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1524-9050&rft.eissn=1558-0016&rft.volume=20&rft.issue=2&rft.spage=727&rft_id=info:doi/10.1109%2FTITS.2018.2829165&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1524-9050&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1524-9050&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1524-9050&client=summon |