Production of bioproducts and biofuels from biomass and waste