How Nintendo Switch's Controllers Track Movement
Compared to the likes of PlayStation and Xbox video game consoles, Nintendo's console offerings have never been at the pinnacle of things like graphical fidelity or processing power. However, what Nintendo consoles lack in raw muscle, they make up for in innovation. For years now, Nintendo has been an industry leader in innovating new ways for players to interact and interface with its games, to the point that even its competitors have tried their hardest to ape its success.
Nowhere is this innovative spirit clearer than with the Nintendo Switch's primary controllers, the Joy-Cons. The obvious gimmick of the Joy-Cons is that you can freely slot or remove them from the Switch proper to create a control style tailored to your tastes, but that's not where things end. The Joy-Cons are packing several forms of specialized hardware for the express purpose of accurately tracking a player's movements and interactions, and translating those movements into gameplay.
Motion controls
The first and most obvious movement-related feature in the Joy-Cons is, of course, motion controls. The Joy-Cons aren't Nintendo's first rodeo in the motion control pen — anyone who's been around for long enough remembers the success of the Nintendo Wii and its waggling Wii Remote. The Joy-Cons possess similar tech to the Wii Remote, albeit to a more advanced degree.
The Joy-Cons' motion control setup is made up of two primary components- a gyro sensor and an accelerometer. The gyro sensor detects the orientation of the controller, whether you're holding it vertically, horizontally, upside-down, and so on. The accelerometer, meanwhile, detects physical movement. As you wiggle, swing, or sway the controller, the accelerometer picks up the speed and intensity of the movement. With these two systems working in tandem, a Joy-Con can detect when it's being swung, the direction it's being swung in, and how hard it's being swung. Each individual Joy-Con possesses these systems, which is how you can have a pair of them detecting motion inputs independently of one another.
While it's not necessarily a matter of motion input, we should also mention the Joy-Cons' HD rumble, which allows it to produce subtle, tailored haptic feedback depending on what's happening in a game. Alongside the movement-detection systems, you could swing a Joy-Con like a sword, and through the HD rumble, feel it realistically clang against a hard surface.
IR motion camera
The movement-tracking feature in the Joy-Cons that you may not necessarily notice at a glance is hidden at the bottom of the right-hand Joy-Con. If you look at the bottom, you'll see a small black rectangle. That's not a decoration, that's the infrared motion camera, or IR camera for short. The IR camera allows the Joy-Con to detect both motion and distinctive images in front of it. If you held your hand in front of the IR camera, for instance, it would be able to see how big your hand is, how far it is from the lens, and its general shape.
The IR camera gives the Joy-Con some extra functionality in the Switch's more physically-involved games like "1-2-Switch" or "Ring Fit Adventure." In the former case, the IR camera captures subtle hand movements for its various minigames, while in the latter, it is used to measure your pulse after a workout by covering it with your thumb. One particularly clever usage of the IR camera was demonstrated in the Nintendo Labo cardboard construction kits. The moving pieces of these kits would feature reflective marker stickers that would bounce the IR camera's light back at it, indicating their presence even in pitch darkness. In the Robot kit, for example, the string pulleys in the backpack were marked with stickers, showing their movements to the IR camera, which then translated those movements to in-game robot controls.