Heat the olive oil and butter in a large pan over a medium heat. Add the walnut halves and the brown sugar, and toss them together for a few minutes, until caramelised. Stir in the cayenne pepper and rosemary, and season with Maldon salt and pepper. Stir everything together gently over a medium heat for a further 1–2 minutes, then tip into a bowl and serve.