Intense light-matter interactions have revolutionized our ability to probe and manipulate quantum systems at sub-femtosecond time scales, opening routes to all-optical control of electronic currents in solids at petahertz rates, the time-scale of a single optical cycle. Such control typically requires electric field amplitudes ~V/A, when the voltage drop across a lattice site becomes comparable to the characteristic band gap energies. In this regime, intense light-matter interaction induces significant modifications of electronic and optical properties and is certain to dramatically modify the band structure of the light-dressed crystal. Yet, identifying and characterizing such modifications remains an outstanding problem. As the oscillating electric field changes from V/A to zero within a quarter-cycle of the driving field, does the band-structure follow, and how can it be defined? Here we address this fundamental question, proposing all-optical spectroscopy of strongly driven crystals, and probe laser-induced closing of the band-gap between adjacent conduction bands. Our work reveals the link between extreme nonlinear light matter interactions in strongly driven crystals to the sub-cycle modifications in their effective band structure.