15:01:19 <quaid> #startmeeting oVirt Infra Team Weekly
15:01:19 <ovirtbot> Meeting started Mon Jan 14 15:01:19 2013 UTC.  The chair is quaid. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:19 <ovirtbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:01:23 <quaid> yay
15:01:30 <quaid> #topic Agenda & roll call
15:01:39 * quaid finds the right channel and starts things ...
15:01:45 <quaid> http://www.ovirt.org/Infrastructure_team_meetings#2012-01-14
15:02:08 <quaid> * Hosting
15:02:08 <quaid> * Puppet
15:02:08 <quaid> * Jenkins
15:02:08 <quaid> * Gerrit
15:02:08 <quaid> * Other business?
15:02:10 <quaid> * Trac review
15:02:14 <quaid> <eoagenda>
15:02:53 <quaid> sorry I'm so bad about reminders for this one, it comes so early for me on Monday that unless I remember to do the reminder on Sunday night ...
15:03:26 <dneary> Morning all
15:04:06 <dneary> re Other business: Add website space issue & adding redundancy for OpenShift administration, please
15:04:33 <dneary> (actually, I guess it's afternoon for most of us - morning Karsten :-) )
15:04:51 <quaid> ping for meeting: ewoud RobertM dcaro eedri
15:05:04 <dcaro> quaid, pong
15:05:36 <quaid> dneary: added, thanks
15:05:56 <quaid> ok, just a few of us, we'll move quickly then :)
15:06:00 <quaid> dcaro: g'day
15:06:10 <quaid> #topic Hosting
15:06:53 <quaid> #info I had a kickoff meeting last Friday with RackSpace for the new hosting there; eedri & dcaro were included (either in the meeting or the materials)
15:07:11 <quaid> #info 10 to 15 business days from Friday 11 Jan for the new hosts to arrive
15:08:42 <quaid> I'll post the details about those hosts to the etherpad
15:08:45 <quaid> http://etherpad.ovirt.org/p/new_hosting_design_Jan_2013
15:09:13 <quaid> #action Need to follow up with Kevin from Alter Way for status
15:09:44 <goacid> Hi :)
15:09:53 <goacid> When you spoke of me
15:10:10 <goacid> Server are beeing installed and will be delivered on wednesday
15:11:36 <quaid> goacid: oh hi!
15:11:42 <goacid> Hi :)
15:11:52 <quaid> #info AlterWay servers are being installed and will be delivered on Wednesday
15:12:02 <dneary> quaid, Would also like to add eedri's request that we add dcaro as Jenkins admin for the master and slaves (and get him to help migrate to the new server).
15:12:04 <quaid> goacid: great, thanks
15:12:16 <quaid> dneary: +1
15:12:20 <dneary> quaid, Given your last info, it's timely :-)
15:12:41 <goacid> DNS will be setup in few minutes with the good reverse
15:12:51 <quaid> dneary: was that just a request of eedri's looking for group +1 or something further needed?
15:13:07 <quaid> iirc, eedri is able to grant those rights, yes?
15:13:49 <quaid> goacid: I was thinking of having the hosts go by the unimaginative alterway{01,02}.ovirt.org
15:14:28 <goacid> Well that is ok for me :) If you want more funny name juste let me know
15:15:27 <quaid> yeah, we can name hosts after spells from Harry Potter :)
15:15:37 * quaid just happy for clear and meaningful
15:15:55 <quaid> ok, we have a hosting design document that is coming together fairly well
15:16:07 <quaid> we can make a plan from it
15:16:44 <quaid> (or modify what is there, that is)
15:17:29 <quaid> I'll work on improving that one today, then throw the plan to the list
15:17:38 <quaid> #action quaid to work on final migration plan for list review
15:17:46 <dcaro> I'm also ok with the boring names
15:17:51 <quaid> anything else for hosting?
15:18:09 * eedri here
15:18:14 <quaid> howdy eedri
15:18:25 <eedri> hey, sorry for being away... had to take care of something
15:19:00 <eedri> +1 on the alterway news...
15:19:19 <quaid> np
15:19:41 <eedri> quaid, i can grant access to jenkins ui
15:19:56 <eedri> quaid, what is the official way to grant ssh sudo access?
15:20:33 <quaid> eedri: honestly, I don't think we did decide anything official :) but ...
15:20:42 <quaid> eedri: is there an open question on the mailing list?
15:20:47 * quaid hasn't read all his email :)
15:21:03 <quaid> I think a few +1s from existing Infra folks and no -1 is the usual way
15:21:15 <quaid> so +1 from me
15:21:19 <eedri> quaid, i meant the technical way
15:21:22 <quaid> ah!
15:21:30 <eedri> quaid, of adding users access
15:21:46 <quaid> I have a wiki page that explains my method I've been using, I think
15:22:10 <quaid> http://www.ovirt.org/Adding_a_new_system_administrator_to_a_host
15:22:47 <dneary> quaid, I imagined that it would be in AOB, but didn't want to forget it. eedri just sent an email to infra proposing dcaro as Jenkins admin.
15:22:58 <dneary> +1 from me
15:23:20 <dneary> I'd also like to hear from ewoud
15:23:34 <quaid> great, we can +1 on list to make it formal
15:24:03 <dcaro> thanks :)
15:24:05 <quaid> ok, we'll bring this up again in a few minutes, do the final approval on the mailing list ASAP
15:24:13 <quaid> going to move on to Puppet ...
15:24:25 <quaid> #topic Puppet
15:24:41 <quaid> #info dcaro has sent initial Puppet manifests to the list
15:25:08 <quaid> http://lists.ovirt.org/pipermail/infra/2013-January/001799.html
15:25:15 <quaid> https://github.com/david-caro/puppet-jenkins
15:25:35 <quaid> I have zero experience with these
15:26:34 <quaid> dcaro: can you contact ewoud to make sure he saw & can reply to that thread?
15:27:59 <quaid> #action Infra team needs especially ewoud need to review initial Jenkins Puppet manifests
15:28:06 <quaid> #undo
15:28:06 <ovirtbot> Removing item from minutes: <MeetBot.items.Action object at 0x93f538c>
15:28:09 <quaid> that was sloppy :)
15:28:13 <dcaro> yes, he already commented some things in the pull request: https://github.com/rtyler/puppet-jenkins/pull/40
15:28:23 <quaid> #action Infra team especially ewoud need to review initial Jenkins Puppet manifests
15:28:29 <quaid> oh, great
15:28:55 <quaid> sweet, that's underway then
15:29:29 <quaid> dcaro: are we going to try to do our config for the AlterWay hosts using Puppet?
15:29:45 <quaid> meaning ... do we have anywhere to put Puppet master?
15:30:40 <quaid> checking the design doc, we have it slated to be a VM on alterway02
15:30:47 <quaid> so ... chicken & egg problem
15:31:02 <eedri> quaid, i think we said we'll install jenkins manually 1st
15:31:16 <eedri> quaid, and after we'll have puppetmaster install we can enforce the puppet classes on it
15:31:27 <eedri> quaid, and make sure it's stays aligned to the configuration
15:31:39 <quaid> ah, yes, thanks!
15:31:44 <quaid> I do recall that
15:32:15 <quaid> #agreed We'll install Jenkins manually, then have puppet master enforce the puppet classes and make sure it stays aligned to the configuration
15:32:31 <quaid> anything else for Puppet topic?
15:33:27 <eedri> quaid, are we going to have it on gerrit?
15:33:35 <eedri> quaid, a puppet git repo
15:33:51 <eedri> quaid, imported from github, that means
15:34:02 <dcaro> eedri, using submodules?
15:34:31 <quaid> makes sense to me
15:34:38 <eedri> dcaro, not necessarily, just to better manage it internally in ovirt
15:34:43 <eedri> dcaro, and not on github
15:35:01 <quaid> +1 to puppet git repo on gerrit
15:35:11 <eedri> dcaro, we can keep having it on github, and cherry pick what we want to use to gerrit.ovirt.org
15:35:19 <dcaro> +1
15:35:22 <eedri> +1
15:35:27 <eedri> ewoud, ?
15:36:17 <quaid> I think I recall when discussing this similar topic, we wanted to make sure we were packaging some of our configs - is that correct?
15:38:21 <quaid> http://docs.puppetlabs.com/guides/best_practices.html
15:38:23 <dneary> dcaro, Did you commit your private RSA key to github?
15:38:33 <dneary> https://github.com/david-caro/puppet-jenkins/tree/master/files
15:39:04 * quaid hates when he does that
15:39:09 <dcaro> nono, it's a custom generated one, only for that, actually you are supposed to change it (I'm still trying to figure out how to avoid having it there)
15:39:55 <quaid> eedri: would you be able to setup the repo on gerrit?
15:40:17 <eedri> quaid, i don't have access/perms to add new repos, only itamar
15:40:27 <quaid> ok
15:40:43 <itamar> quaid, which one?
15:40:46 <quaid> eedri: can you bring up the request on infra@ and we'll get ewoud's input there?
15:40:48 <dneary> Ah :-)
15:40:49 <ale> sorry if i should interrupt the meeting / need to know though how to detach an export domain ? do i need to set the DC into maintenance mode ? if i do that - does anything happen to the VMs running ?
15:40:54 <dneary> dcaro, This makes sense
15:40:58 <dneary> Excuse me!
15:41:45 <quaid> itamar: I think we're talking about one or more new repos for Infra, this one for Puppet manifests
15:41:57 * quaid doesn't know the answer to ale's question
15:42:02 <itamar> please send an email to infra mailing list (cc me) asking for it creation...
15:42:12 <quaid> itamar: +1
15:42:49 <quaid> actually ... dcaro can you make that request on infra@ and we'll get final input there?
15:43:58 <dcaro> quaid: sure
15:44:10 <ale> i tried copying the exported templates to another export domain, but it complains that the templates already exist in the system ... :\ .... so it does find the VM Templates but i cannot import them into the new DC
15:44:46 <quaid> #action dcaro to request Puppet module on Gerrit
15:45:32 <quaid> #agreed Puppet repo on Gerrit generally a good idea, will finish detail discussions on list
15:45:36 <quaid> moving on ...
15:45:48 <quaid> #topic Jenkins
15:46:07 <quaid> #action All reply to the on-list request to grant admin access to dcaro
15:46:51 <quaid> #info Jenkins migration plan embedded in the overall new hosting design
15:46:53 <quaid> http://etherpad.ovirt.org/p/new_hosting_design_Jan_2013
15:47:04 <quaid> anything else on Jenkins here?
15:47:33 <eedri> recently we had some failures on jenkins jobs, it's motly cause of infra
15:47:44 <eedri> i suspect slow vm response
15:47:50 <eedri> and out of space issues
15:48:05 <eedri> hopefully will be solved once we'll move to new infra
15:48:19 <quaid> definitely hoping so :)
15:48:49 <quaid> #info recent job failure on Jenkins are likely due to infrastructure - slow VM response and out of space issues
15:49:29 <quaid> any more on Jenkins?
15:49:44 <quaid> topic going once ....
15:49:56 <quaid> topic going twice ....
15:50:01 <eedri> sold
15:50:03 <quaid> #topic Gerrit
15:50:23 <quaid> #info Gerrit is generally stable currently
15:50:37 <quaid> previously we agreed we'll be planning that move after we get Jenkins et al settled
15:51:15 <quaid> #agreed Still planning to move Gerrit to new hosting after things settle down from the Jenkins and Mailman moves
15:51:40 <quaid> #idea Should do the Gerrit install using the new Foreman/Puppet system
15:51:50 * quaid knows that's an obvious one, just getting it in the record :)
15:52:09 <quaid> ok, going to close this one quickly ... anything more?
15:53:18 <vfeenstr> um HTTPS on gerrit?
15:53:41 <quaid> ah
15:53:52 <quaid> I think we agreed to do that, yes
15:54:11 <quaid> #action quaid to get an SSL Cert for one or more ovirt.org domains
15:54:22 <quaid> I'll make sure we have a ticket open for that, and start working it
15:55:12 * vfeenstr luckily looked into here at the right moment :)
15:55:18 <quaid> I want to get this one last topic in before the hour
15:55:20 * ewoud here now
15:55:23 <ewoud> stupid meetings
15:55:23 <quaid> so I'll do that ticket in parallel
15:55:34 <quaid> #topic OpenShift - space & admin redundancy
15:56:00 <quaid> #info we ran out of space on our OpenShift gear over the weekend, which causes an unhelpful error that dneary fixed by cleaning up things
15:56:12 <quaid> #action quaid is asking #openshift for a much bigger quota
15:56:22 <dneary> I just heard back from the OpenShift guys
15:56:24 <quaid> dneary: what were you thinking about admin redundancy?
15:56:40 <dneary> We have a bigger quota (did you see Grant Shipley's email?
15:57:13 <dneary> quaid, I learned that this is as simple as "rhc log-clean" or something like that
15:57:36 <dneary> quaid, So - you're the creator of the "app", I only have access through git
15:57:59 <dneary> I asked if there was any way for me to access admin tasks using rhc, and that's only possible for the app creator
15:58:04 <dneary> Apparently
15:58:11 <quaid> dneary: didn't see that, thanks
15:58:20 <quaid> #info Bigger quota is now here, thanks dneary
15:58:23 <dneary> Just figuring out how we can make sure that we don't need you if there is a problem :-)
15:58:31 <quaid> #info 'rhc log-clean' helps keep things clean
15:58:42 <quaid> #idea can we automate openshift cleanup using cron?
15:59:10 <dcaro> is there any kind of log rotation policy? I don't like the idea of having to run the cleanup from another machine
15:59:13 <quaid> #action quaid needs to figure out how to add other admins to OpenShift, then add folks from Infra
15:59:43 <quaid> #idea can we use OpenShift log rotation automagic?
16:00:10 <dneary> dcaro, These are the questions to which I do not have answers
16:00:16 <quaid> me either :)
16:00:33 <dneary> I did google the error message I got, and apparently "we're working on a better solution" is the answer people got in forums
16:00:35 <quaid> but we'll get some
16:01:23 <dcaro> I found someone suggesting using the cron cartdrige for that
16:01:42 <quaid> #idea could use cron cartridge for our own log rotation
16:01:55 <eedri> mburns, ping
16:02:42 <mburns> eedri: pong
16:02:49 <myoung34> so i wrote a little script a while back to generate a spice link for all my vm's using ovirt-shell, and all-uva-sudden it's complaining that --password is no longer allowed...do i have to generate keys to get this working again?
16:03:37 <myoung34> basically it ran list vms, then looped through them and generated output like vmname: ___ port: ___ password: ___ to make my life easier
16:03:53 <quaid> ok, I've got another meeting about to distract me, so ....
16:04:04 <quaid> anything else before we close this topic and/or the meeting?
16:04:30 <ale> :| .. any1 know what happens to the VMs if I put a DC into Maintenance ? Are they being stopped / or you just cant do anything while the host is in Maint mode ?
16:05:32 <ale> it'd be cool if a DC  could use multiple export domains that do not belong to the DC ..
16:06:30 <quaid> closing meeting in one minutes
16:08:49 <quaid> Thanks folks!
16:08:53 <quaid> #endmeeting