In understanding the nature and the formation of glasses lie questions that go beyond the physics of such amorphous materials and are relevant for many branches of science from physics to mathematics, computer science, and biology. After a general introduction to the field of glassy systems, I will show that strong non-linear responses to small perturbations and the proliferation of soft-modes are key signatures of glassiness. I will explain how we used these features to probe the nature of the glass transition and to reveal that amorphous solids may undergo a new kind of phase transition at which standard elasticity breaks down.