Hyperdimensional computing (HDC) is an emerging computing paradigm that exploits the distributed representation of input data in a hyperdimensional space, the dimensions of which are typically between 1,000--10,000. The hyperdimensional distributed representation enables energy-efficient, low-latency, and noise-robust computations with low-precision and basic arithmetic operations. In this study, we propose optical hyperdimensional distributed representations based on laser speckles for adaptive, efficient, and low-latency optical sensor processing. In the proposed approach, sensory information is optically mapped into a hyperdimensional space with >250,000 dimensions, enabling HDC-based cognitive processing. We use this approach for the processing of a soft-touch interface and a tactile sensor and demonstrate to achieve high accuracy of touch or tactile recognition while significantly reducing training data amount and computational burdens, compared with previous machine-learning-based sensing approaches. Furthermore, we show that this approach enables adaptive recalibration to keep high accuracy even under different conditions.