Many real-world systems require experimentation in some form, where the decision to expend resources is based on a cost-benefit trade-off. This study examines the effect of various challenges of optimal decision-making during experimental resource allocation. The study particularly draws attention to characteristics such as the effects of resources, treatments, temporal changes, external interventions and human feedback. The study assumes a Bayesian approach where the strengths of factor effects have distributional priors, and the response depends linearly on the factors. In this study, we address three specific challenges. First, we look at quantifying the expected benefit of exploring a design space, which can be conceptualized as the response of the best treatment combination in a full factorial experiment. Second, we look at a special case where the decision is a go/no-go with a single treatment, and the effect of resources/experimental units can be learned from optimizing the benefit gained from experimenting on different strata of units. Third, we look at the temporal effect on the resource allocation between treatments, which may be applicable to systems that require frequent updates based on the recent past. While many existing solutions in the literature assume the system/process to be stationary, we address the problem of optimizing the resource allocation in experiments for a given time period in non-stationary systems, particularly in an online environment. With a continuous form of experimentation and exploitation of prior knowledge, the maximization of expected benefit would lead to a steady state for the resource allocation.