Merged changes from trunk to correctly support ONE_AUTH and the OpenNebula authorization file.
Additional stats printing functions
Added option to interrupt simulation after a given amount of (simulated) time has passed.
- Added support for lease annotation files where the annotations are not bound to a specific lease- haizea-lwf-stats now prints a summary of requested images- Tweaked discrete distributions in stats module
Utilization-based LWF generation (still needs some work, but mostly done)
LWF generator mostly working
Beginnings of haizea.lwf package
Move suspension/resumption time estimation methods from VMScheduler to Lease
- Don't allow distributions to share state.- Added truncated pareto distribution
Miscellaneous minor bug fixes.
Collect some statistics when converting a SWF file to LWF.
Add support for annotation files in multiconfig files.
Finished annotator command
Cleaned up stats module (still needs documentation). Added bounded Pareto distribution.Added tests for stats module
Created branch for 1.2 beta (doing it right this time)
A few minor changes so Haizea will work with Python 2.4
Switched to a saner implementation for Singleton classes. Required multiple changes all around (which had the indirect side effect of making constructors less dependent on the get_config() function)
Miscellaneous pylint-induced code cleanup.
Added script to automatically generate probe reference appendix.
Made persistence configurable. Documented persistence code.
- Data persistence is implemented. It still needs some work (and has to be made configurable so it can be switched off during simulations), but it's working.- Started to clean up the accounting code so it will collect data using "probes" instead of hard-coded statements.
Updated copyright notices.
Merged TP2.0/0.9 branch into trunk.
Cleaned up code and added more comments, primarily in LeaseScheduler
Sanitized and improved exception handling
Keep track of lease state using a state machine. Added more sanity checks that throw exceptions when lease is in an inconsistent state.
Started refactoring scheduling code into more manageable modules + classes. Note that this leaves some parts of the scheduler broken for the time being.
Beginnings of Lease state machine
Apparently I got my branches messed up and I don't know how to do a proper merge yet... :-P
- Started removing lots of unnecessary dependencies, mostly references from objects to containing objects, which should be avoided. In particular, lots of objects don't have a "convenient" rm attribute pointing back to the Resource Manager. This attribute was only used to access the configuration file, the accounting object, and the clock. So, these three have been made "global" (a little globality is not bad, as long as it's controlled; in this case, the only way of accessing these objects is using the get_config, get_accounting, and get_clock global functions which, in turn, access the ResourceManager singleton)....
Add option to stop Haizea (in OpenNebula mode) when there are no leases left to process.
Refactored new suspend/resume code
Miscellaneous code cleaning, most of it required by the previous two revisions.
Changed the topology of the lease data structure. Instead of containing a flat list of resource reservations, it now separates between deployment RRs and VM RRs, with the latter now being able to contain multiple suspend/resume RRs and migration RRs. This is necessary to accomodate complex suspend/resume scenarios (e.g., having multiple VMs on a physical node, saving memory files to a global filesystem, etc.)
Added defaults module to contain default values, instead of hardcoding them throughout the code
Changed logging code so there will be multiple loggers, accessible through logging.getLogger, instead of a single logger in the ResourceManager class (which required traversing the class hierarchy to get to that logger). Also made ResourceManager a singleton.
Adapt multiconfig to new configuration management. Save multiconfig attributes to accounting data object.
Overhauled configuration file management. Lots of minor changes in most modules on account of this.
Added RPC/CLI for listing leases and cancelling leases
- Made clock configurable in simulation (i.e., user can choose the simulated clock that was being used so far, or a real clock where time is "real" and everything else is simulated)- Moved wakeup-interval from opennebula section to scheduling section
Added "wakeup-interval" and "non-schedulable-interval" to [opennebula] section (these options were previously hardcoded)
- Added haizea-convert-data command to convert Haizea's data file to more useable formats (barebones for now)- Fix: actual starting and end times were not being saved (which did not allow waiting time and slowdown to be computed)- Removed painfully old analysis modules.
- Cleaned up stats.py- Divided haizea.resourcemanager.stats.Stats into StatsData and StatsCollection. The former only contains the data (and will soon include convenient get methods) and the latter is in charge of collecting the data throughout Haizea's run....
Reorganized CLI code
- Immediate lease scheduling- Signal handling to gracefully stop on SIGTERM or SIGINT- Added "onevm" option to config file (location of onevm command)
- Cleaned up datastruct module (docstrings still missing). Multiple modules have been updated since so much stuff depends on the datastruct classes.- Added "preemptability" attribute to LeaseBase. Updated ResourceManager.__schedule_besteffort accordingly....
- First stab at decoupling deployment scheduling code from the notion of "image transfers". Not ideal yet, but at least a lot of the image transfer scheduling code has been moved out of the Scheduler class and into a delegate class in the haizea.resourcemanager.deployment package...
bugfix: getInjectfile was returning image file, not the inject file
Remove unused imports
Miscellaneous code cleanup
- Made logging messages in "info" level a bit more useful.- Miscellaneous name changes
Remove old code
Re-added support for multi-configuration files
Make some of the option names more descriptive
Writing collected statistics to disk
Fixed whitespace in license notice
Update copyright information
Reimplemented stats class to be more general and extensible.
Removed obsolete multirun.py module
Added license information to Python files
Rename "exact lease" to "AR lease" (the term used in the paper)
Bugfix of the bugfix
- Fixed bug in date rounding (breaks when rounding second >59.5)- Replaced convoluted UNIX2DateTime with simpler one.
- Round times in real clock- Allow real clock to fastforward
Decouple suspend/resume rate from simulation
Modified ResourceTuple structure to make it more flexible (no longer hardcoded to have 5 resource typesin a specific order; allows for enactment backends to add more attributes for matchmaking beyond cpu, mem, and disk).
Added function to convert UNIX timestamps to mxDateTime (needed by OpenNebula frontend)
Beginning of OpenNebula enactment (CLI-based, not XML-RPC yet). This required modifying some code that was still coupled to the simulation code.
Improved clock interfaces and added real clock.
Improved organization of logging code.
New resource pool module and enactment modules that depend on it. Only the simulation module is (partially) implemented.
Broke down "general" and "simulation" configuration sections into "general", "simulation", "scheduling" and "tracefile".
Reorganized code, mostly factoring out code from ResourceManager. Notable changes:- Using Clock class, allowing for a real clock to be eventually used.- Trace reading code moved to frontend.- Everything from main.py folded back into other classes.
Default value of suspend threshold should be 0
Change in package names
Directory reorganization
Removed old EARS code
Make suspend threshold a floating point number
Beginnings of making suspend threshold a configurable knob
Make "avoid preemption" policy toggleable
Add option to generate scripts only for experiments with missing data (useful to generate condor files just for the experiments that have failed and have to be rerun)
[Small fixes] Make reporting tool more modular, so report + graph generation can be split into multiple smaller processes (that can be run in parallel)
Make reporting tool more modular, so report + graph generation can be split into multiple smaller processes (that can be run in parallel)
(minor fixes) Produce reports+graphs that show correlation between a metric and lease characteristics
Produce reports+graphs that show correlation between a metric and lease characteristics
Remove hardcoding of graphs that will be produced in a report, and allow this to be specified in the experiment config file
- Save completed lease data for use in reporting- Changes to clipping in reporting
More optimizations
More optimizations and minor fixes
Better data structures in slot table, and other optimizations
First stab at new implementation of slot table
- Generate normalized injections- Add option to stop simulator when last best-effort lease is submitted- Small bugfixes
Added bounded slowdown metric
Image reuse for best-effort leases
- Purging unused images from image pools- Image reuse for exact leases
- Produce disk usage statistics in results- Keep better track of pool images and tainted images
- Added image reuse infrastructure to simulated backend- Improved file handling in simulated backend (distinguish between reusable images, tainted images, and RAM files)