Majid Raeis

ITSC 2021
[Reinforcement learning, Fairness, Traffic signal control]

In recent years, traffic control methods based on deep reinforcement learning (DRL) have gained attention due to their ability to exploit real-time traffic data, ...which is often poorly used by the traditional hand-crafted methods. While most recent DRL-based methods have focused on maximizing the throughput or minimizing the average travel time of the vehicles, the fairness of the traffic signal controllers has often been neglected. This is particularly important as neglecting fairness can lead to situations where some vehicles experience extreme waiting times, or where the throughput of a particular traffic flow is highly impacted by the fluctuations of another conflicting flow at the intersection. In order to address these issues, we introduce two notions of fairness: delay-based and throughput-based fairness, which correspond to the two issues mentioned above. Furthermore, we propose two DRL-based traffic signal control methods for implementing these fairness notions, that can achieve a high throughput as well. Read more

AAAI 2021
[Reinforcement learning, QoS, Queueing theory]

End-to-end delay is a critical attribute of quality of service (QoS) in application domains such as cloud computing and computer networks. This metric ...is particularly important in tandem service systems, where the end-to-end service is provided through a chain of services. Service-rate control is a common mechanism for providing QoS guarantees in service systems. In this paper, we introduce a reinforcement learningbased (RL-based) service-rate controller that provides probabilistic upper-bounds on the end-to-end delay of the system, while preventing the overuse of service resources. Read more

ICAPS 2021
[Distributed scheduling, Fairness, Multi-agent systems]

Information exchange is a crucial component of many realworld multi-agent systems. However, the communication between the agents involves two major challenges: the limited bandwidth, and the shared communication medium between the agents, which ...restricts the number of agents that can simultaneously exchange information. While both of these issues need to be addressed in practice, the impact of the latter problem on the performance of the multi-agent systems has often been neglected. In this work, we propose a distrbuted fair scheduling algorithm, called distributed self-clocked fair queueing (DSCFQ), to deal with this issue. Read more

IEEE GLOBECOM 2020
[Admission control, Queueing networks, Reinforcement learning, Delay-sensitive applications]

Ensuring quality of service (QoS) guarantees in service systems is a challenging task, particularly when the system is composed of more fine-grained services, such as service function chains. An important QoS metric ...in service systems is the end-to-end delay, which becomes even more important in delay-sensitive applications, where the jobs must be completed within a time deadline. Admission control is one way of providing end-to-end delay guarantee, where the controller accepts a job only if it has a high probability of meeting the deadline. In this paper, we propose a reinforcement learning-based admission controller that guarantees a probabilistic upper-bound on the end-to-end delay of the service system, while minimizes the probability of unnecessary rejections. Read more

ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), 2020
[Queueing theory, Exponential martingale, Energy storage]

This article presents an analysis of a recently proposed queueing system model for energy storage with discharge. Even without a load, ...energy storage systems experience a reduction of the stored energy through self-discharge. We consider a queueing model, referred to as leakage queue, where, in addition to an arrival and a service process, there is a leakage process that reduces the buffer content by a factor γ (0 < γ < 1) in each time slot. We derive expressions for the probabilities of overflow and underflow of the leakage queue (battery), which correspond to the probabilities of having wasted energy and lost demand, respectively. Read more

Drones have been considered as an alternative means of package delivery to reduce the delivery cost and time. Due to the battery limitations, the drones are best suited for last-mile delivery, i.e., the delivery from the package distribution centers (PDCs) to the customers. Since a typical delivery system ...consists of multiple PDCs, each having random and time-varying demands, the dynamic drone-to-PDC allocation would be of great importance in meeting the demand in an efficient manner. In this paper, we study the dynamic UAV assignment problem for a drone delivery system with the goal of providing measurable Quality of Service (QoS) guarantees. Read more