Many real-world systems require experimentation in some form, involving a commitment of resources to different treatments under study. The decision to expend resources is based on a cost-benefit analysis. This research examines the effect of various challenges in optimal resource allocation strategies in non-standard online experimental settings. We use a Bayesian framework where the factor effects have distributional priors, and the response depends linearly on the factors. In our first study, we investigate the worth of conducting an experiment that is applicable to both offline and online settings. We derive the distribution of the maximum expected benefit gained from exploring a design space with distributional prior beliefs of the factor effects. Given the potential advantages of conducting experiments, we discuss some associated challenges, including the impact of treatments, resources, and temporal changes. Extant literature proposes models that assume a standard multi-treatment framework for stationary experimental settings. In our second study, we flip this to model a non-standard online environment that fixes a single treatment and experiments in the space of heterogeneous resources. By maximizing the expected benefit from two phases of one-shot exploration followed by selective exploitation, we recommend the optimal allocation to different strata of the resources. In our third study, we determine the effect of temporal changes and treatments in a non-stationary online experiment. Owing to the inherent dynamic nature, we assume a moving-window setup where the decision of choosing the better treatment among two alternatives at a particular time period depends only on the last few periods. With a continuous form of experimentation and exploitation of prior knowledge, maximization of the expected benefit would lead to a steady state for the resource allocation. We show applications of the above studies based on experimental data from FinTech and provide ideas for further deployment.