14:00:27 <quaid> #startmeeting
14:00:27 <ovirtbot> Meeting started Tue Aug 14 14:00:27 2012 UTC.  The chair is quaid. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:27 <ovirtbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:00:40 <quaid> #meetingname Infra Team weekly sync
14:00:40 <ovirtbot> The meeting name has been set to 'infra_team_weekly_sync'
14:00:52 <quaid> #topic roll call & agenda
14:00:58 * quaid is here :)
14:01:07 * mburns lurking, but distracted
14:01:56 * RobertM here
14:02:19 <quaid> quiet today
14:02:42 <quaid> ''Agenda''
14:02:42 <quaid> * Post-release check-in
14:02:42 <quaid> * Puppet
14:02:42 <quaid> * Hosting search
14:02:42 <quaid> * Jenkins
14:02:44 <quaid> * All other business
14:03:21 * eedri here
14:03:30 <quaid> ok, let's go ahead with topics
14:03:37 <quaid> #topic Post-release check-in
14:04:00 <quaid> Anything noteworthy from last week's release for this team?
14:04:38 <RobertM> quaid, Other then the fact that the new structure was in place and worked fine.
14:05:03 <quaid> web traffic to ovirt.org was less than for the 3.0 release, so no risk of overwhelming the services
14:05:45 <quaid> tracking and reporting on those sort of trends (downloads, sit visits, etc.) is part of my $dayjob, btw
14:06:18 <quaid> and I'll be doing all the tooling & how-to within oVirt directly, so we'll all be able to watch the same trends
14:06:52 <quaid> anything we noticed to consider for next release?
14:07:20 <RobertM> quaid, What tracker are you using?
14:07:33 <quaid> I realized that we were doing changes on infra right up to the release; Fedora Infra for example has change freeze for some weeks leading up to the release
14:07:44 <quaid> we may want to consider something similar as the project grows
14:07:51 <RobertM> quaid, No I think we implemented the changes that were needed to make 3.2 go smoothly.
14:08:04 <eedri> quaid, +1 on that
14:08:27 <eedri> quaid, probably focus on monitor and fix and not push new features around releases
14:08:46 <quaid> RobertM: combo is: google analytics, mlstats + small custom subscriber count script, and still need to find something good for Apache logs, that's about it
14:08:57 <RobertM> quaid, +1 but part of the reason is the group really didn't form until right before release.
14:09:06 <quaid> yep
14:09:39 <quaid> I don't think we were ever at risk, it's just a better practice, where we can do it
14:09:49 <RobertM> We addressed the issue with the repo after beta2.
14:10:06 <quaid> right
14:10:17 <RobertM> We should be in much better shape come 3.2
14:10:44 <quaid> RobertM: do you agree that we'd normally want to have a change freeze for some $time_period leading up to the release
14:11:32 <RobertM> quaid, Yes
14:11:34 <mburns> change yes, but net-new services should be ok as long as they don't interact with the existing stuff
14:11:53 * ewoud should really set up a notification for this meeting
14:13:00 <RobertM> quaid, But we have to be flexable.  Since releases themselves can require a change.  AKA the repo structure had to change to make room for 3.1
14:13:22 <quaid> #agreed We'll set some light-but-flexible policy about having a change-to-existing-infra freeze for a week (or similar time period) before a releases
14:13:35 <RobertM> It should have been caught and fixed before had but sometimes stuff like that will come out of no where.
14:13:48 <quaid> right
14:14:09 <quaid> we can also just use a change process, as a group we can approve doing changes in the freeze window
14:14:43 <quaid> the idea is to just have a time where we are more careful than normal just-in-case, and the care is more about checking with others to be sure we aren't risking breaking something on release
14:15:57 <RobertM> logical
14:16:59 <quaid> anything more?
14:17:12 <RobertM> No
14:17:37 <RobertM> Should add that to the release process in the wiki
14:19:17 <quaid> #action add change notice procedure process to release process in the wiki
14:19:26 <quaid> #topic Puppet
14:19:40 <ewoud> I think that's my cue
14:19:49 <ewoud> you've seen my proposal on the list?
14:20:32 <ewoud> In short: I wrote some puppet classes to manage a slave and it's available my github
14:20:35 <ewoud> #link https://github.com/ekohl/ovirt-infra-puppet
14:21:08 <ewoud> opinions?
14:21:43 <ewoud> so far on the ML I saw eedri and quaid respond with some suggestions, anyone else?
14:21:46 <RobertM> ewoud, Are there diff classes for Fedora and EL?
14:21:48 <quaid> looks like we have some discussion going there, which is good
14:22:15 <ewoud> RobertM: not yet, but we can build those when needed
14:23:21 <eedri> ewoud, not sure we want to have everything under one module
14:23:27 <eedri> ewoud, ovirt-infra-puppet
14:23:49 <eedri> ewoud, might be better to have small and atomic class for each purpose
14:23:53 <ewoud> eedri: the current repo is meant as a parent git repo which you use as an environment
14:24:22 <eedri> ewoud, the repo is fine, i'm talking abot the moudle 'ovirt_infra'
14:24:29 * quaid puts some comments directly in to the thread
14:25:02 <eedri> ewoud, or at least as subclasses
14:25:18 <eedri> ewoud, for example instead of having  package {['vim-enhanced', 'git']: in the main ovirt_infra
14:25:36 <eedri> ewoud, put it in ovirt_infa::required_pkg..
14:25:48 <eedri> ewoud, but i think the best approach is to create a class per resource
14:26:00 <eedri> ewoud, so you would create a class for 'git' seperatly
14:26:22 <ewoud> eedri: I agree, that would be better
14:26:26 <eedri> ewoud, i know it's more work, but i found it to be a much more flexible way of building complex hostgroups in the future
14:26:47 <ewoud> eedri: I certainly want to split the users into their own classes so you can give a user permissions to a server etc
14:28:16 <ewoud> but to continue working on this I think we need to set some goals
14:28:51 <ewoud> I think first should be to set up some infra to collaborate on it, so that should be a puppet master and a gerrit repo
14:29:28 <ewoud> and the puppet master can be either pure puppet or foreman, where the latter could certainly provide some cool features
14:30:24 <ewoud> would the current kitchen sink have sufficient memory to run foreman?
14:30:54 <quaid> don't know
14:31:09 <quaid> can Foreman run on e.g. OpenShift?
14:31:31 <eedri> ewoud, i agree that we should do a basic design or requirment doc before writing puppet classes
14:31:42 <eedri> ewoud, maybe start a wiki page on it..
14:31:53 <ewoud> eedri: I will do that
14:31:57 <eedri> ewoud, great
14:32:16 <quaid> +1
14:32:25 <eedri> ewoud, once we'll get an agreement from the team on the basic classes we can start writing them
14:32:27 <ewoud> quaid: I think you're the only chair so can you fix a #action?
14:32:49 <eedri> ewoud, maybe you should open a new project in gerrit for puppet classes?
14:32:55 <eedri> ewoud, instead of git hum
14:33:12 <quaid> ewoud: anyone can do action, fwiw
14:33:23 <quaid> #chair ewoud eedri mburns RobertM
14:33:23 <ovirtbot> Current chairs: RobertM eedri ewoud mburns quaid
14:33:29 <ewoud> #action ewoud write a wiki page on puppet
14:33:58 <quaid> (I think only #agreed and something else I forget are limited to chair, plus some actions such as giving out chair, opening or closing the meeting, etc.)
14:34:14 <ewoud> eedri: a gerrit project is git and I certainly do want to continue there
14:35:07 <RobertM> I think github is fine for the working group when we go to implentant we will want to move to gerrit.
14:35:13 <quaid> I posed some more questions/comments to the thread, maybe we can continue from there?
14:35:28 <ewoud> RobertM: yes
14:35:30 <quaid> RobertM: yeah, do we have an Infra git repo yet?
14:35:35 <eedri> RobertM, yes, i agree that github is usefull for the devel phase..
14:35:41 <RobertM> +1 for continue on thread
14:35:44 <quaid> would this be a stand-alone repo, or part of a big Infra repo?
14:35:51 <eedri> RobertM, once we have all the classes we want we can push it to gerrit infra project
14:35:57 * quaid actually asked this in the thread, so will take his answers there
14:36:10 <quaid> ok, good stuff!
14:36:22 <quaid> going to be very exciting to have Puppet (and Foreman) running)
14:36:38 <quaid> anything more on this topic here?
14:36:40 <eedri> quaid, as long as we'll have the power (vms) to run them properly :)
14:37:14 <RobertM> eedri, That is a very good point.
14:37:57 <quaid> yes
14:38:10 <quaid> well, that's an upcoming topic, so let's move there
14:38:31 <quaid> #topic Hosting search
14:39:18 <ewoud> anyone looked for hosting?
14:39:22 <quaid> last week I got some basic budget approval, I think we can work with US$150/mon for hosting
14:39:27 <quaid> but I haven't done any searching beyond that
14:39:53 <quaid> also, not sure what we want - 5 VMs with that? One bare metal server with massive specs so we can run on it and put up our own VMs on it?
14:40:18 <quaid> anyone interested in leading the specs and research here?
14:40:35 <quaid> we could be running within a few days, but that research needs to be done to get there
14:40:51 * quaid can help but hasn't been good at leading the research so far
14:40:54 <RobertM> I would like to be part of the group since I have been having so much fun with Jenkins I have a feel for what it requires.
14:41:22 <quaid> can we just do this from a new mailing list thread?
14:41:43 <ewoud> ok, RobertM will you propose something on the ML?
14:41:55 <RobertM> ewoud, ok
14:42:51 <quaid> #action RobertM will kick off the hosting proposal thread, where we can do resarch
14:42:53 <ewoud> RobertM: would be interested in $ vs euro and also US-based hosting or EU-based, can you include those?
14:43:02 <quaid> good point
14:43:20 <RobertM> I assume are budget is in US $
14:43:24 <quaid> I'll be paying either with credit card or (preferably) a purchase order
14:43:29 <quaid> RobertM: yes
14:43:30 <RobertM> are=our
14:44:06 <RobertM> So the week dollar will make euro hasting more expensive.
14:44:12 <quaid> #info Looking at October++ for Red Hat IT to have possible VMs for us to use permanently, which would replace this paid-for hosting
14:44:17 <RobertM> *weak
14:44:32 <quaid> RobertM: maybe we can find a US-based company with EU data centers :)
14:45:23 <ewoud> so next agenda item?
14:45:46 <ewoud> #topic Jenkins
14:46:14 <RobertM> Were do we start with Jenkins :)
14:46:27 * ewoud is just following the agenda
14:47:10 <RobertM> eedri, Have you had a change to look into the nightly and package generation?
14:47:29 <eedri> RobertM, sorry, didn't have time this week
14:47:46 <eedri> RobertM, but i think we might need to ask on ovirt weekly
14:48:13 <eedri> RobertM, about if a code change is needed in the various projects for supporting rpm names
14:48:23 <eedri> RobertM, like it is supported in ovirt-node and vdsm
14:48:58 <RobertM> eedri, Do you want to take care of bring that up on the Wednesday meeting?
14:49:05 <eedri> RobertM, ok
14:49:55 <ewoud> but do I understand it correct that we currently have 2 jenkins masters?
14:50:14 <RobertM> ewoud, Yes.
14:50:36 <eedri> #action eedri to bring up makefile changes in MAKEFILE for ovirt projects to support per-commit rpm names
14:51:03 <RobertM> ovirt.org had the master branch dealing with building nightly.
14:51:19 <RobertM> ovirt.info is running the per patch stuff.
14:52:21 <RobertM> jenkins.ovirt.org and jenkins.ovirt.info
14:54:27 <quaid> wow, is that really all we have for Jenkins today?
14:54:33 * quaid *shocked*
14:54:39 <RobertM> Per Patch jobs are running for both VDSM and Engine on jenkins.ovirt.info
14:54:59 <quaid> #info Per Patch jobs are running for both VDSM and Engine on jenkins.ovirt.info
14:55:17 <quaid> are we fine with having two masters for the moment?
14:55:37 <eedri> RobertM, we need to verify all jobs are ineed runs on the relevant refspec
14:55:38 <RobertM> quaid, We really don't have much of a choice.
14:55:47 <eedri> RobertM, some users told me it wasn't the case for some patches..
14:55:49 <RobertM> eedri, Yes we do
14:56:15 <ewoud> quaid: I think there are some slaves under each master
14:56:32 <ewoud> and I'd prefer to move to a single master or at least a single infra
14:56:33 <eedri> RobertM, i think its due to wrong 'git strategy' for some jobs
14:56:50 <eedri> ewoud, the secondary master was build primarly as a backup server
14:56:54 <eedri> ewoud, and testing server
14:56:58 <RobertM> eedri, I went though them all last night.  But I would love a 2nd set of eyes
14:57:28 <eedri> ewoud, the only reason that per patch jobs are running on it is that the master can't handle it (the ec2)
14:57:29 <ewoud> eedri: I know, but I mean a single primary for production which makes it easier to replicate a staging env
14:57:47 <eedri> ewoud, true, would be possible once we'll have a proper master
14:58:21 <eedri> ewoud, i don't think that the master will be able to handle all the patch jobs. even if they run on a slave
14:59:14 <ewoud> eedri: even on a decent server?
14:59:26 <eedri> ewoud, no, on a decent server it will manage
14:59:35 <quaid> first thing with new hosting will be to move Jenkins master(s) over, yes?
14:59:46 <eedri> ewoud, i have a jenkins master with more than 50 slaves
14:59:47 <RobertM> ewoud, jenkins.ovirt.info was doing the same jobes as jenkins.ovirt.org in half the time but it is way backed up.  With the right hardware we can merge the 2 into 1 master.
15:00:17 <ewoud> ok, so jenkins improvements are pending new hosting now
15:00:18 <eedri> ewoud, a proper master can defiently handle the load i foresee for ovirt project for the coming future
15:00:34 <quaid> #agreed First use of new hosting is to be combined Jenkins master
15:00:53 <RobertM> +1
15:01:03 <eedri> ewoud, it would just need slaves to running jobs
15:01:19 <ewoud> anything else about jenkins or can we move on?
15:01:25 <quaid> if we want to setup puppet.ovirt.org on the Linode box ... we could possibly use it to build the new Jenkins master, yes?
15:01:35 <quaid> ewoud: +1
15:01:37 <eedri> i think the whitelist has been set up quite well also
15:01:49 <ewoud> quaid: yes, could certainly help
15:01:50 <eedri> need to pay attention for users that are missing from it
15:02:31 <eedri> ewoud, there is a puppet moudle for jenkins http://puppetlabs.com/blog/module-of-the-week-rtylerjenkins-continuous-integration-server/
15:02:39 <ewoud> eedri: I know, and I've used it
15:02:44 <eedri> ewoud, :)
15:02:56 <ewoud> eedri: but then you informed me that a slave doesn't need to run it, so I removed it ;)
15:03:01 <quaid> anything more?
15:03:06 <RobertM> Building the new master based on puppet will be a good way to test the classes and find the large number of missing packages that are needed that aren't in there already
15:03:10 <eedri> ewoud, yea.. only the master ;)
15:03:20 <ewoud> I'd like to discus some sort of task tracker
15:03:27 <eedri> +1 on traker
15:03:37 <ewoud> doesn't have to be fixed in the short term, but things are getting harder to track
15:03:49 <eedri> RobertM, we'll probably have 2 profiles - 1 for master and 1 for slave
15:03:54 <eedri> RobertM, for starts
15:04:27 <RobertM> Depends on if we are running jobes on the master or not.
15:04:54 <ewoud> quaid: do you have any experience or know someone who has experience with a task/issue tracker for us?
15:05:34 <RobertM> ewoud, quaid Do we have a preference on what task tracker to use.  Those things tend to be personal prefer things
15:05:46 <quaid> ewoud: I was going to recommend we use Teambox
15:06:01 <quaid> #topic Task trackers
15:06:28 <quaid> Teambox is hosted and open source
15:06:35 <quaid> (the latest version isn't yet opened, but will be)
15:06:53 <quaid> folks I know who use it really like it
15:06:56 <ewoud> quaid: how open can we make it?
15:07:31 <ewoud> i.e. read only access to anonymous users?
15:07:34 <quaid> otherwise, we pick another hosting or *shudder* host it ourselves
15:08:03 <quaid> ewoud: I think so, but we want to be sure it doesn't require an account/login to view, worth double-checking
15:08:35 <ewoud> quaid: can you work out a proposal for the mailing list?
15:09:39 <quaid> #action quaid to research Teambox, potential propose it as a new thread on task tracking
15:10:58 <ewoud> Anything else?
15:11:05 <ewoud> nothing more on the agenda
15:11:30 <ewoud> quaid: RobertM eedri?
15:11:44 <ewoud> closing in 30 seconds
15:12:32 <ewoud> nothing?
15:12:35 <ewoud> going once
15:12:38 <ewoud> going twice
15:12:41 <ewoud> #endmeeting