This research pioneers formal combined plant and controller optimization (termed ``co-design") schemes for complex systems where accurate but expensive experiments are fused with cheap but less accurate numerical simulations. This fusion of experiments with numerical modeling tools, referred to as experimentally infused optimization, addresses several challenges faced in co-design research. These challenges include strong coupling between the controller and plant, significant modeling uncertainties (which require the use of experiments), and high experimental costs. This research presents two unique iterative experimentally infused optimization approaches: One approach uses small ``batches" of experiments at each iteration to identify the gradient of a performance index, whereas the other approach leverages tools from optimal design of experiments and hypothesis testing to efficiently explore and reduce the design space at each iteration. Both approaches incorporate a model correction term that improves the numerical model prediction after each set of experiments. For the latter approach, an original theoretical convergence bound on the numerical model improvement in the presence of noise-influenced experiments is presented and derived in this dissertation. Furthermore, both approaches have been validated for performance-critical parameters on a lab-scale airborne wind energy (AWE) system test platform at UNC Charlotte. Dimensional analysis has been used to show that this unique lab-scale platform produces results that replicate the full-scale flight behavior under uniformly-scaled system time constants. Overall, the quality of flight for the AWE system is greatly improved using experimental infusion as compared to a pure numerical optimization approach.