Yes, you can use pavers to build an outdoor pizza oven for fun family dinners. However, if you want to get serious about your pizza making, its better to use specially designed fire bricks. Fire bricks retain heat better, withstand higher temperatures, and will end up baking your pizza better.