Dr. Andreas Sedlmeier

Dr. Andreas Sedlmeier Lehrstuhl für Mobile und Verteilte Systeme Ludwig-Maximilians-Universität München, Institut für Informatik Oettingenstraße 67 80538 München Raum E 008 Telefon: +49 89 / 2180-9173 Fax: +49 89 / 2180-9148 Mail: andreas.sedlmeier@ifi.lmu.de

Research Interests

• Artificial Intelligence
• Machine Learning
• Uncertainty and Robustness in AI Systems
• Reinforcement Learning

Publications

2022

• F. Ritz, T. Phan, A. Sedlmeier, P. Altmann, J. Wieghardt, R. Schmid, H. Sauer, C. Klein, C. Linnhoff-Popien, and T. Gabor, „Capturing Dependencies Within Machine Learning via a Formal Process Model,“ in Leveraging Applications of Formal Methods, Verification and Validation. Adaptation and Learning, 2022, p. 249–265. doi:10.1007/978-3-031-19759-8_16
  [BibTeX] [Abstract] [Download PDF]
  

  The development of Machine Learning (ML) models is more than just a special case of software development (SD): ML models acquire properties and fulfill requirements even without direct human interaction in a seemingly uncontrollable manner. Nonetheless, the underlying processes can be described in a formal way. We define a comprehensive SD process model for ML that encompasses most tasks and artifacts described in the literature in a consistent way. In addition to the production of the necessary artifacts, we also focus on generating and validating fitting descriptions in the form of specifications. We stress the importance of further evolving the ML model throughout its life-cycle even after initial training and testing. Thus, we provide various interaction points with standard SD processes in which ML often is an encapsulated task. Further, our SD process model allows to formulate ML as a (meta-) optimization problem. If automated rigorously, it can be used to realize self-adaptive autonomous systems. Finally, our SD process model features a description of time that allows to reason about the progress within ML development processes. This might lead to further applications of formal methods within the field of ML.

  @InProceedings{ritz22capturing,
  author ="Ritz, Fabian and Phan, Thomy and Sedlmeier, Andreas and Altmann, Philipp and Wieghardt, Jan and Schmid, Reiner and Sauer, Horst and Klein, Cornel and Linnhoff-Popien, Claudia and Gabor, Thomas",
  editor = "Margaria, Tiziana and Steffen, Bernhard",
  title = "Capturing Dependencies Within Machine Learning via a Formal Process Model",
  booktitle = "Leveraging Applications of Formal Methods, Verification and Validation. Adaptation and Learning",
  year = "2022",
  publisher = "Springer Nature Switzerland",
  pages = "249--265",
  isbn = "978-3-031-19759-8",
  doi = "10.1007/978-3-031-19759-8_16",
  url = "https://arxiv.org/abs/2208.05219",
  abstract = "The development of Machine Learning (ML) models is more than just a special case of software development (SD): ML models acquire properties and fulfill requirements even without direct human interaction in a seemingly uncontrollable manner. Nonetheless, the underlying processes can be described in a formal way. We define a comprehensive SD process model for ML that encompasses most tasks and artifacts described in the literature in a consistent way. In addition to the production of the necessary artifacts, we also focus on generating and validating fitting descriptions in the form of specifications. We stress the importance of further evolving the ML model throughout its life-cycle even after initial training and testing. Thus, we provide various interaction points with standard SD processes in which ML often is an encapsulated task. Further, our SD process model allows to formulate ML as a (meta-) optimization problem. If automated rigorously, it can be used to realize self-adaptive autonomous systems. Finally, our SD process model features a description of time that allows to reason about the progress within ML development processes. This might lead to further applications of formal methods within the field of ML."
  }

• F. Ritz, T. Phan, R. Müller, T. Gabor, A. Sedlmeier, M. Zeller, J. Wieghardt, R. Schmid, H. Sauer, C. Klein, and C. Linnhoff-Popien, „Specification Aware Multi-Agent Reinforcement Learning,“ in Agents and Artificial Intelligence, 2022, p. 3–21. doi:10.1007/978-3-031-10161-8_1
  [BibTeX] [Abstract]
  

  Engineering intelligent industrial systems is challenging due to high complexity and uncertainty with respect to domain dynamics and multiple agents. If industrial systems act autonomously, their choices and results must be within specified bounds to satisfy these requirements. Reinforcement learning (RL) is promising to find solutions that outperform known or handcrafted heuristics. However in industrial scenarios, it also is crucial to prevent RL from inducing potentially undesired or even dangerous behavior. This paper considers specification alignment in industrial scenarios with multi-agent reinforcement learning (MARL). We propose to embed functional and non-functional requirements into the reward function, enabling the agents to learn to align with the specification. We evaluate our approach in a smart factory simulation representing an industrial lot-size-one production facility, where we train up to eight agents using DQN, VDN, and QMIX. Our results show that the proposed approach enables agents to satisfy a given set of requirements.

  @InProceedings{ritz22specification,
  author = "Ritz, Fabian and Phan, Thomy and M{\"u}ller, Robert and Gabor, Thomas and Sedlmeier, Andreas and Zeller, Marc and Wieghardt, Jan and Schmid, Reiner and Sauer, Horst and Klein, Cornel and Linnhoff-Popien, Claudia",
  editor = "Rocha, Ana Paula and Steels, Luc and van den Herik, Jaap",
  title = "Specification Aware Multi-Agent Reinforcement Learning",
  booktitle = "Agents and Artificial Intelligence",
  year = "2022",
  publisher = "Springer International Publishing",
  pages = "3--21",
  isbn = "978-3-031-10161-8",
  doi = "10.1007/978-3-031-10161-8_1",
  abstract = "Engineering intelligent industrial systems is challenging due to high complexity and uncertainty with respect to domain dynamics and multiple agents. If industrial systems act autonomously, their choices and results must be within specified bounds to satisfy these requirements. Reinforcement learning (RL) is promising to find solutions that outperform known or handcrafted heuristics. However in industrial scenarios, it also is crucial to prevent RL from inducing potentially undesired or even dangerous behavior. This paper considers specification alignment in industrial scenarios with multi-agent reinforcement learning (MARL). We propose to embed functional and non-functional requirements into the reward function, enabling the agents to learn to align with the specification. We evaluate our approach in a smart factory simulation representing an industrial lot-size-one production facility, where we train up to eight agents using DQN, VDN, and QMIX. Our results show that the proposed approach enables agents to satisfy a given set of requirements."
  }

• A. Sedlmeier, M. Kölle, R. Müller, L. Baudrexel, and C. Linnhoff-Popien, „Quantifying Multimodality in World Models,“ in Proceedings of the 14th International Conference on Agents and Artificial Intelligence – Volume 1: ICAART,, 2022, pp. 367-374. doi:10.5220/0010898500003116
  [BibTeX]

  @conference{multimod_icaart22,
  author = {Andreas Sedlmeier and Michael Kölle and Robert Müller and Leo Baudrexel and Claudia Linnhoff-Popien},
  title = {Quantifying Multimodality in World Models},
  booktitle = {Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
  year = {2022},
  pages = {367-374},
  publisher = {SciTePress},
  organization = {INSTICC},
  doi = {10.5220/0010898500003116},
  isbn = {978-989-758-547-0}
  }

2021

• S. Illium, R. Müller, A. Sedlmeier, and C. Popien, „Visual Transformers for Primates Classification and Covid Detection,“ in Proc. Interspeech 2021, 2021, p. 451–455. doi:10.21437/Interspeech.2021-273
  [BibTeX]

  @inproceedings{illium21_interspeech,
  author = {Steffen Illium and Robert Müller and Andreas Sedlmeier and Claudia-Linnhoff Popien},
  title = {{Visual Transformers for Primates Classification and Covid Detection}},
  year = 2021,
  booktitle = {Proc. Interspeech 2021},
  pages = {451--455},
  doi = {10.21437/Interspeech.2021-273}
  }

• F. Ritz, T. Phan, R. Müller, T. Gabor, A. Sedlmeier, M. Zeller, J. Wieghardt, R. Schmid, H. Sauer, C. Klein, and C. Linnhoff-Popien., „SAT-MARL: Specification Aware Training in Multi-Agent Reinforcement Learning,“ in Proceedings of the 13th International Conference on Agents and Artificial Intelligence – Volume 1: ICAART,, 2021, pp. 28-37. doi:https://doig.org/10.5220/0010189500280037
  [BibTeX] [Abstract] [Download PDF]
  

  A characteristic of reinforcement learning is the ability to develop unforeseen strategies when solving problems. While such strategies sometimes yield superior performance, they may also result in undesired or even dangerous behavior. In industrial scenarios, a system’s behavior also needs to be predictable and lie within defined ranges. To enable the agents to learn (how) to align with a given specification, this paper proposes to explicitly transfer functional and non-functional requirements into shaped rewards. Experiments are carried out on the smart factory, a multi-agent environment modeling an industrial lot-size-one production facility, with up to eight agents and different multi-agent reinforcement learning algorithms. Results indicate that compliance with functional and non-functional constraints can be achieved by the proposed approach.

  @conference{ritz21satmarl,
  author = {Fabian Ritz and Thomy Phan and Robert Müller and Thomas Gabor and Andreas Sedlmeier and Marc Zeller and Jan Wieghardt and Reiner Schmid and Horst Sauer and Cornel Klein and Claudia Linnhoff-Popien.},
  title = {SAT-MARL: Specification Aware Training in Multi-Agent Reinforcement Learning},
  booktitle = {Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
  year = {2021},
  pages = {28-37},
  publisher = {SciTePress},
  organization = {INSTICC},
  doi = {https://doig.org/10.5220/0010189500280037},
  isbn = {978-989-758-484-8},
  url = {https://arxiv.org/abs/2012.07949},
  abstract = {A characteristic of reinforcement learning is the ability to develop unforeseen strategies when solving problems. While such strategies sometimes yield superior performance, they may also result in undesired or even dangerous behavior. In industrial scenarios, a system’s behavior also needs to be predictable and lie within defined ranges. To enable the agents to learn (how) to align with a given specification, this paper proposes to explicitly transfer functional and non-functional requirements into shaped rewards. Experiments are carried out on the smart factory, a multi-agent environment modeling an industrial lot-size-one production facility, with up to eight agents and different multi-agent reinforcement learning algorithms. Results indicate that compliance with functional and non-functional constraints can be achieved by the proposed approach.}
  }

• T. Phan, L. Belzner, T. Gabor, A. Sedlmeier, F. Ritz, and C. Linnhoff-Popien, „Resilient Multi-Agent Reinforcement Learning with Adversarial Value Decomposition,“ in 35th AAAI Conference on Artificial Intelligence (AAAI 2021), 2021, p. 11308–11316.
  [BibTeX] [Download PDF]

  @inproceedings{phan2021resilient,
  title = {Resilient Multi-Agent Reinforcement Learning with Adversarial Value Decomposition},
  author = {Phan, Thomy and Belzner, Lenz and Gabor, Thomas and Sedlmeier, Andreas and Ritz, Fabian and Linnhoff-Popien, Claudia},
  booktitle = {35th AAAI Conference on Artificial Intelligence (AAAI 2021)},
  volume = {35},
  number = {13},
  pages = {11308--11316},
  year = {2021},
  url = {https://ojs.aaai.org/index.php/AAAI/article/view/17348},
  owner = {tphan}
  }

2020

• S. Illium, R. Müller, A. Sedlmeier, and C. Linnhoff-Popien, „Surgical Mask Detection with Convolutional Neural Networks and Data Augmentations on Spectrograms,“ in Proc. Interspeech 2020, 2020, p. 2052–2056. doi:10.21437/Interspeech.2020-1692
  [BibTeX] [Download PDF]

  @inproceedings{Illium2020,
  author = {Steffen Illium and Robert Müller and Andreas Sedlmeier and Claudia Linnhoff-Popien},
  title = {{Surgical Mask Detection with Convolutional Neural Networks and Data Augmentations on Spectrograms}},
  year = 2020,
  booktitle = {Proc. Interspeech 2020},
  pages = {2052--2056},
  doi = {10.21437/Interspeech.2020-1692},
  url = {http://dx.doi.org/10.21437/Interspeech.2020-1692}
  }

• A. Sedlmeier, R. Müller, S. Illium, and C. Linnhoff-Popien, „Policy Entropy for Out-of-Distribution Classification,“ in Artificial Neural Networks and Machine Learning – ICANN 2020, Cham, 2020, p. 420–431.
  [BibTeX] [Abstract]
  

  One critical prerequisite for the deployment of reinforcement learning systems in the real world is the ability to reliably detect situations on which the agent was not trained. Such situations could lead to potential safety risks when wrong predictions lead to the execution of harmful actions. In this work, we propose PEOC, a new policy entropy based out-of-distribution classifier that reliably detects unencountered states in deep reinforcement learning. It is based on using the entropy of an agent’s policy as the classification score of a one-class classifier. We evaluate our approach using a procedural environment generator. Results show that PEOC is highly competitive against state-of-the-art one-class classification algorithms on the evaluated environments. Furthermore, we present a structured process for benchmarking out-of-distribution classification in reinforcement learning.

  @inproceedings{sedlmeier2020peoc,
  author = {Sedlmeier, Andreas
  and Müller, Robert
  and Illium, Steffen
  and Linnhoff-Popien, Claudia},
  editor = {Farka{\v{s}}, Igor
  and Masulli, Paolo
  and Wermter, Stefan},
  title = {Policy Entropy for Out-of-Distribution Classification},
  booktitle = {Artificial Neural Networks and Machine Learning -- ICANN 2020},
  year = {2020},
  publisher = {Springer International Publishing},
  address = {Cham},
  pages = {420--431},
  abstract = {One critical prerequisite for the deployment of reinforcement learning systems in the real world is the ability to reliably detect situations on which the agent was not trained. Such situations could lead to potential safety risks when wrong predictions lead to the execution of harmful actions. In this work, we propose PEOC, a new policy entropy based out-of-distribution classifier that reliably detects unencountered states in deep reinforcement learning. It is based on using the entropy of an agent's policy as the classification score of a one-class classifier. We evaluate our approach using a procedural environment generator. Results show that PEOC is highly competitive against state-of-the-art one-class classification algorithms on the evaluated environments. Furthermore, we present a structured process for benchmarking out-of-distribution classification in reinforcement learning.},
  isbn = {978-3-030-61616-8}
  }

• T. Phan, T. Gabor, A. Sedlmeier, F. Ritz, B. Kempter, C. Klein, H. Sauer, R. Schmid, J. Wieghardt, M. Zeller, and others, „Learning and Testing Resilience in Cooperative Multi-Agent Systems,“ in Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, 2020, p. 1055–1063.
  [BibTeX] [Download PDF]

  @inproceedings{phan2020learning,
  title = {Learning and Testing Resilience in Cooperative Multi-Agent Systems},
  author = {Phan, Thomy and Gabor, Thomas and Sedlmeier, Andreas and Ritz, Fabian and Kempter, Bernhard and Klein, Cornel and Sauer, Horst and Schmid, Reiner and Wieghardt, Jan and Zeller, Marc and others},
  booktitle = {Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems},
  pages = {1055--1063},
  year = {2020},
  url = {http://ifaamas.org/Proceedings/aamas2020/pdfs/p1055.pdf},
  owner = {tphan}
  }

• C. Hahn, T. Phan, S. Feld, C. Roch, F. Ritz, A. Sedlmeier, T. Gabor, and C. Linnhoff-Popien, „Nash Equilibria in Multi-Agent Swarms,“ in 12th International Conference on Agents and Artificial Intelligence (ICAART 2020), 2020.
  [BibTeX]

  @inproceedings{hahn2020nash,
  author = {Carsten Hahn and Thomy Phan and Sebastian Feld and Christoph Roch and Fabian Ritz and Andreas Sedlmeier and Thomas Gabor and Claudia Linnhoff-Popien},
  title = {Nash Equilibria in Multi-Agent Swarms},
  booktitle = {12th International Conference on Agents and Artificial Intelligence (ICAART 2020)},
  year = {2020},
  owner = {chahn}
  }

• A. Sedlmeier, T. Gabor, T. Phan, L. Belzner, and C. Linnhoff-Popien, „Uncertainty-Based Out-of-Distribution Classification in Deep Reinforcement Learning,“ in 12th International Conference on Agents and Artificial Intelligence (ICAART 2020), 2020.
  [BibTeX]

  @inproceedings{sedlmeier2020UBOOD,
  author = {Andreas Sedlmeier and Thomas Gabor and Thomy Phan and Lenz Belzner and Claudia Linnhoff-Popien},
  title = {Uncertainty-Based Out-of-Distribution Classification in Deep Reinforcement Learning},
  booktitle = {12th International Conference on Agents and Artificial Intelligence (ICAART 2020)},
  year = {2020},
  owner = {chahn}
  }

2019

• M. Friedrich, F. Guimera Cuevas, A. Sedlmeier, and A. Ebert, „Evolutionary Generation of Primitive-Based Mesh Abstractions,“ in Proceedings of the 27th International Conference on Computer Graphics, Visualization and Computer Vision (WSCG), 2019.
  [BibTeX]

  @inproceedings{friedrich2019generation-mesh,
  author = {Friedrich, Markus and Guimera Cuevas, Felip and Sedlmeier, Andreas and Ebert, André},
  title = {Evolutionary Generation of Primitive-Based Mesh Abstractions},
  booktitle = {Proceedings of the 27th International Conference on Computer Graphics, Visualization and Computer Vision (WSCG)},
  year = {2019},
  owner = {mfriedrich}
  }

• T. Gabor, A. Sedlmeier, M. Kiermeier, T. Phan, M. Henrich, M. Picklmair, B. Kempter, C. Klein, H. Sauer, R. Schmid, and J. Wieghardt, „Scenario Co-Evolution for Reinforcement Learning on a GridWorld Smart Factory Domain,“ in Genetic and Evolutionary Computation Conference (GECCO), 2019.
  [BibTeX]

  @inproceedings{Gabor2019a,
  author = {Thomas Gabor and Andreas Sedlmeier and Marie Kiermeier and Thomy Phan and Marcel Henrich and Monika Picklmair and Bernhard Kempter and Cornel Klein and Horst Sauer and Reiner Schmid and Jan Wieghardt},
  title = {Scenario Co-Evolution for Reinforcement Learning on a GridWorld Smart Factory Domain},
  booktitle = {Genetic and Evolutionary Computation Conference (GECCO)},
  year = {2019},
  publisher = {ACM},
  __markedentry = {[gruttaue:6]},
  owner = {tgabor}
  }

• S. Feld, A. Sedlmeier, M. Friedrich, J. Franz, and L. Belzner, „Bayesian Surprise in Indoor Environments,“ in 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL ’19), 2019.
  [BibTeX]

  @inproceedings{feld2019bayesian,
  author = {Feld, Sebastian and Sedlmeier, Andreas and Friedrich, Markus and Franz, Jan and Belzner, Lenz},
  title = {Bayesian Surprise in Indoor Environments},
  booktitle = {27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL ’19)},
  year = {2019},
  organization = {ACM}
  }

• A. Sedlmeier, T. Gabor, T. Phan, and L. Belzner, „Uncertainty-Based Out-of-Distribution Detection in Deep Reinforcement Learning,“ in 1st International Symposium on Applied Artificial Intelligence (ISAAI’19), 2019.
  [BibTeX]

  @inproceedings{sedlmeier2019uncertainty,
  author = {Andreas Sedlmeier and Thomas Gabor and Thomy Phan and Lenz Belzner},
  title = {Uncertainty-Based Out-of-Distribution Detection in Deep Reinforcement Learning},
  booktitle = {1st International Symposium on Applied Artificial Intelligence (ISAAI'19)},
  year = {2019},
  owner = {asedlmeier}
  }

2018

• A. Sedlmeier and S. Feld, „Discovering and Learning Recurring Structures in Building Floor Plans,“ in LBS 2018: 14th International Conference on Location Based Services, 2018, p. 151–170.
  [BibTeX]

  @inproceedings{sedlmeier2018discovering,
  author = {Sedlmeier, Andreas and Feld, Sebastian},
  title = {Discovering and Learning Recurring Structures in Building Floor Plans},
  booktitle = {LBS 2018: 14th International Conference on Location Based Services},
  year = {2018},
  pages = {151--170},
  publisher = {Springer},
  owner = {asedlmeier}
  }

• T. Gabor, M. Kiermeier, A. Sedlmeier, B. Kempter, C. Klein, H. Sauer, R. Schmid, and J. Wieghardt, „Adapting Quality Assurance to Adaptive Systems: The Scenario Coevolution Paradigm,“ in International Symposium on Leveraging Applications of Formal Methods (ISoLA), 2018.
  [BibTeX]

  @inproceedings{gabor2018adapting,
  title = {Adapting Quality Assurance to Adaptive Systems: The Scenario Coevolution Paradigm},
  author = {Thomas Gabor and Marie Kiermeier and Andreas Sedlmeier and Bernhard Kempter and Cornel Klein and Horst Sauer and Reiner Schmid and Jan Wieghardt},
  booktitle = {International Symposium on Leveraging Applications of Formal Methods (ISoLA)},
  year = {2018}
  }

• S. Feld, S. Illium, A. Sedlmeier, and L. Belzner, „Trajectory annotation using sequences of spatial perception,“ in 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL ’18), 2018, p. 1–10. doi:https://doi.org/10.1145/3274895.3274968
  [BibTeX]

  @inproceedings{feld2018trajectory,
  author = {Feld, Sebastian and Illium, Steffen and Sedlmeier, Andreas and Belzner, Lenz},
  title = {Trajectory annotation using sequences of spatial perception},
  booktitle = {26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL ’18)},
  year = {2018},
  pages = {1--10},
  organization = {ACM},
  doi = {https://doi.org/10.1145/3274895.3274968}
  }

• A. Sedlmeier and S. Feld, „Learning indoor space perception,“ Journal of Location Based Services, pp. 1-36, 2018. doi:10.1080/17489725.2018.1539255
  [BibTeX] [Download PDF]

  @article{sedlmeier2018learning,
  author = {Andreas Sedlmeier and Sebastian Feld},
  title = {Learning indoor space perception},
  journal = {Journal of Location Based Services},
  year = {2018},
  volume = {0},
  number = {0},
  pages = {1-36},
  doi = {10.1080/17489725.2018.1539255},
  owner = {asedlmeier},
  url = {https://doi.org/10.1080/17489725.2018.1539255}
  }

2011

• M. Dürr, M. Duchon, K. Wiesner, and A. Sedlmeier, „Distributed Group and Rights Management for Mobile Ad Hoc Networks,“ in 4th Joint IFIP Wireless and Mobile Networking Conference (WMNC 2011), 2011, pp. 1-8. doi:10.1109/WMNC.2011.6097258
  [BibTeX]

  @inproceedings{durr2011b,
  title = {Distributed Group and Rights Management for Mobile Ad Hoc Networks},
  author = {Michael Dürr and Markus Duchon and Kevin Wiesner and Andreas Sedlmeier},
  booktitle = {4th Joint IFIP Wireless and Mobile Networking Conference (WMNC 2011)},
  year = {2011},
  pages = {1-8},
  publisher = {IEEE},
  doi = {10.1109/WMNC.2011.6097258}
  }

Talks

• Talk@Trail 2020: Modelling Uncertainty for Learning Systems (alternative PDF Version)

Preprints / Postprints

• Learning Indoor Space Perception in Journal of Location Based Services, pp. 1-36, 2018.
  This is an Accepted Manuscript of an article published by Taylor & Francis in Journal of Location Based Services on 11/10/2018, available online: https://www.tandfonline.com/10.1080/17489725.2018.1539255.