Abstract: State-of-the-art beat tracking is based on neural networks that model beats and downbeats as individual events, often followed by a separate Dynamic Bayesian Network (DBN) enforcing musical consistency. We propose a novel architecture that combines deep learning with differentiable theory-based modules: Specifically, we design a system that first predicts a tatum grid, then groups tatum events to beats and downbeats, explicitly modelling multiple options for the current tempo and time signature in a bar-pointer-like fashion. Our approach improves continuity metrics compared to existing DBN-free systems and is more versatile than the de-facto standard DBN, but still falls behind the best DBN-based methods on average. Given the many design choices and hyperparameters involved, we assume we have not yet explored the full potential of this idea and publish our code and invite others to explore this further.