15:01:08 <ewoud> #startmeeting oVirt infra weekly
15:01:08 <ovirtbot> Meeting started Mon Feb 25 15:01:08 2013 UTC.  The chair is ewoud. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:08 <ovirtbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:01:19 <ewoud> copy+paste from http://www.ovirt.org/Infrastructure_team_meetings#2013-02-25
15:01:26 <ewoud> Agenda
15:01:30 <ewoud> Hosting Puppet Jenkins Gerrit Other business? Trac review
15:01:55 <ewoud> #chair Rydekull garrett_ dcaro eedri_
15:01:55 <ovirtbot> Current chairs: Rydekull dcaro eedri_ ewoud garrett_
15:01:59 <knesenko> ewoud: what about new guys introduction ? :)
15:02:13 <ewoud> knesenko: I'd like to start with that then :)
15:02:32 <ewoud> so let's
15:02:40 <ewoud> #topic Introductions
15:02:44 <knesenko> Hi all my name is Kiril I am from integration team of oVirt product
15:03:01 <knesenko> obasan: your turn :)
15:03:10 <ewoud> knesenko: did you also start the thread 'New ideas to grow the community'?
15:03:22 <knesenko> ewoud: Yes I am
15:03:38 <obasan> hi my name is Ohad
15:03:53 <obasan> I am from int. team as well .
15:04:11 <Rydekull> For someone who is also quite new, whats the integration team? :-)
15:04:19 <ewoud> Rydekull: I was about to ask :)
15:04:34 <eedri_> Rydekull, i think knesenko meant integration team for the rhevm product
15:04:50 <knesenko> that what I meant
15:05:00 <ewoud> obasan, knesenko: welcome
15:05:07 <dcaro> knesenko, obasan: welcome!!
15:05:10 <mkublin> jhernand, ping
15:05:16 <obasan> ewoud, dcaro thanks :)
15:05:17 <ewoud> anything in particular you're interested in picking up?
15:05:22 <quaid> +1 welcome
15:05:32 <mburns> ewoud: hey, when you get a second, in this meeting, i have some input on the alterway servers
15:05:33 <eedri_> welcome guys
15:05:36 <Rydekull> ewoud: not sure what that means either
15:05:43 <Rydekull> ewoud: err, sorry
15:05:48 <ewoud> mburns: I think we can do that under hosting
15:05:54 <Rydekull> eedri_: not sure what that means either. But, ok :-)
15:06:01 <mburns> ewoud: sure, just ping me when you get there...
15:06:05 <ewoud> mburns: will do
15:06:06 <knesenko> system administration, jenkins, automation jobs - sounds great
15:06:22 <eedri_> Rydekull, in a few words: integration team is in charge of ci testing for rhevm, building rhevm various projects
15:06:36 <ewoud> cool
15:07:00 <Rydekull> Aha
15:07:10 <Rydekull> That explains a bit more :-)
15:07:37 <eedri_> Rydekull, using foreman & puppet of course
15:07:53 <Rydekull> knesenko, obasan: Welcome, glad you're willing to join us :-)
15:08:10 <knesenko> So me and obasan don't have problems to work on jenkins, puppet and systems administration
15:08:15 <knesenko> we are here to contribute
15:08:16 <knesenko> :)
15:08:24 <ewoud> FYI: I'm ewoud, mostly interested in setting up foreman & puppet now
15:08:43 <ewoud> so I'm sure we'll work together
15:08:51 <knesenko> ewoud: np
15:09:02 <ewoud> looking forward to it
15:09:04 <ewoud> anyone else wants to introduce or shall we move on to hosting?
15:09:10 <knesenko> I finished
15:09:15 <ewoud> #topic hosting
15:09:34 <ewoud> #info eedri started on the jenkins migration
15:09:49 <ewoud> #undo
15:09:49 <ovirtbot> Removing item from minutes: <MeetBot.items.Info object at 0xa41f36c>
15:09:56 <eedri_> yes. new server is alive @ alterway01.ovirt.org
15:09:59 <ewoud> #info eedri started on the jenkins migration, details on the ML and migration planned for thursday
15:10:24 <eedri_> ewoud, we might start it on wed if possible even
15:10:33 <mburns> ewoud: can i jump in here before we plan any more migrations or set timelines?
15:10:40 <ewoud> mburns: sure
15:11:05 <mburns> i just talked to acathrow and we can get full RHEL licenses (with support) for the alterway boxes
15:11:22 <quaid> we need those for the RAX boxen
15:11:44 <quaid> i'm trying to do that, took over from dneary
15:11:58 <mburns> quaid: we can probably get them for that too, but i'd need to talk to acathrow
15:12:00 <quaid> then we can have Jenkins slaves there that run RHEL
15:12:00 <ewoud> mburns: I think one reason we chose for centos is that our users are also likely to run centos
15:12:08 <eedri_> mburns, we have a talk about which OS to install on the alterway servers, and we included rhel as one of the options. but we decided on centos
15:12:16 <eedri_> not because we couldn't get rhel
15:12:37 <quaid> yep, we decided also to not keep revisiting the decision
15:13:17 * Rydekull has tabled the question long ago in his mind :-)
15:13:38 <eedri_> it was a very long discussion indeed :)
15:13:38 <mburns> quaid: eedri_ that's fine, i wasn't paying attention to the discussion, and the topic was raised to me directly that we perhaps "should" be using rhel...
15:13:50 <ewoud> mburns: so if you now have 2 licenses then we can use those for RAX, right?
15:14:04 * Rydekull has no clue what RAX is
15:14:06 <quaid> mburns: we should, as well as should be using centos too
15:14:10 <ewoud> rackspace I'm guessing
15:14:11 <quaid> rackspace
15:14:48 <quaid> (stock ticker)
15:14:53 <mburns> ewoud: i don't have anything at the moment, i was just told that we can get supported entitlements for the alterway boxes
15:15:12 <mburns> i expect it's also true of the RAX boxes...
15:15:16 <ewoud> mburns: ok
15:15:35 <ewoud> I hope quaid will talk to the right people to arrange that :)
15:15:37 <mburns> if it's decided that we don't want them on alterway, that's fine, i'll back and tell them
15:15:38 <quaid> mburns: i'm going to include acathrow on my work there, so we can get to this desired state too
15:15:45 <mburns> quaid: ack
15:16:12 <ewoud> #info redhat offered RHEL licences for alterway servers, but it has been decided that we want to run centos
15:16:14 <quaid> mburns: yeah, it's decided and in the ml archives in Dec & Jan
15:16:46 <ewoud> so I think we can start the jenkins migration as soon as eedri_ has time
15:17:18 <eedri_> ewoud, yes. new server is ready, so next in line is the migration. hopefully will start either on wed/thu
15:17:41 <eedri_> ewoud, quaid how do we want to handle the dns change?
15:18:04 <ewoud> eedri_: I don't know since I have no idea how RH DNS works
15:18:26 <ewoud> which may also become relevant when we start using foreman to provision: it would be nice if we could talk to a DNS API somehow
15:18:31 <ewoud> though that's optional
15:18:37 <quaid> hmm
15:19:01 <eedri_> right now i'm under the assumtion that the new server is alterway01.ovirt.org
15:19:09 <quaid> in this case, we want to move jenkins.ovirt.org?
15:19:12 <eedri_> i'm guessing that after the migration will be over and verified, we'll want to change it
15:19:32 <eedri_> maybe change jenkins.ovirt.org to archive-jenkins.ovirt.org for backward recovery
15:19:33 <ewoud> the TTL is already 300, so it should be fast but it should be coordinated
15:19:52 <ewoud> quaid: I think so
15:20:32 <knesenko> eedri_: you can point 2 ips to the same fqdn
15:20:44 <knesenko> eedri_: ignore
15:21:09 <quaid> we can request a specific time window, especially with days to plan
15:21:36 <ewoud> quaid: and if we run into problems, how easy is it to cancel that change?
15:21:56 <dcaro> we can even use a proxy to show the new server from the old one while the dns change applies
15:21:59 <ewoud> what we also can do is set up a reverse proxy on the old host
15:22:04 <ewoud> dcaro: :)
15:22:21 <eedri_> that's great
15:22:30 <eedri_> no need for dns change then
15:22:36 <eedri_> until we'll be sure the new server works
15:22:41 <ewoud> yes
15:22:44 <dcaro> eedri_: yep
15:22:48 <eedri_> _!
15:22:49 <eedri_> +1
15:23:17 <ewoud> #info we'll use a reverse proxy for jenkins.ovirt.org until we've verified the migration is completed, then change DNS
15:23:31 <ewoud> ok, I think that's enough about the jenkins migration
15:23:53 <quaid> ewoud: should be easy to cancel
15:23:57 <eedri_> anyone is working on alterway02?
15:24:03 <YamaKasY> ewoud: mhh ok.. what shall I try ?
15:24:11 <quaid> waiting for centos reinstall?
15:24:27 <ewoud> I saw Rydekull mailed to arch@ just before the meeting, got a +1 but no centos 6 packages yet
15:24:40 <ewoud> YamaKasY: not sure, bit busy too now
15:24:52 <Rydekull> Yeah, I will follow up by trying to contact the specific maintainers directly during this week. I was a bit swamped on this issue
15:25:37 <ewoud> quaid: both blocked on centos 6 reinstall, but after that on EL6 packages
15:25:39 <YamaKasY> ewoud: okay!
15:26:35 <ewoud> given we're blocked I'd propose that we do start setting up a foreman + puppet base
15:27:10 <ewoud> I can update my foreman PoC environment so we at least have a base to start with
15:27:35 <eedri_> ewoud, are we planning to install foreman as a VM on ovirt instance (on alterway02)?
15:27:55 <ewoud> then stuff like creating the user accounts can be set up, inclusing SSH keys; we can play with our puppet repo in gerrit
15:27:59 <ewoud> eedri_: eventually yes
15:28:05 <ewoud> eedri_: but until we
15:28:20 <ewoud> 've arranged that I think we shouldn't block on that completely
15:29:05 <ewoud> #action ewoud update foreman PoC to something functional
15:29:27 <ewoud> I'll work on that we aren't stuck
15:29:45 <ewoud> I think that covers hosting, puppet & jenkins
15:30:05 <ewoud> can we move on to the next point on the agenda?
15:30:11 <Rydekull> +1
15:30:23 <ewoud> #topic gerrit
15:30:44 <knesenko> we are talking about gerrit.ovirt.org ?
15:30:45 <ewoud> about the topic: it's been on our agenda template, but we haven't actually discussed it for a few weeks
15:30:48 <ewoud> knesenko: yes
15:31:15 <ewoud> I think the eventual goal is that we're going to maintain it, but right now it's not really
15:31:28 <eedri_> ewoud, plan is to migrate it also right?
15:31:33 <ewoud> eedri_: correct
15:32:08 <quaid> yes, when we have things stabilized, including team processes sonitMs clear how we support it, etc.
15:32:37 <ewoud> should we leave it off the agenda for next week?
15:32:42 <eedri_> +1
15:33:01 <ewoud> #topic other business
15:33:21 <ewoud> knesenko mentioned http://www.ovirt.org/Becoming_an_Infrastructure_team_member needs some work on the ML
15:33:27 <ewoud> I think he's right
15:34:32 <knesenko> ewoud: We need to create some steps
15:34:38 * Rydekull would be surprised if anyone thought otherwise
15:34:40 <knesenko> enjoying mailing lists
15:34:44 <knesenko> tasks
15:34:47 <knesenko> etc ...
15:35:05 <ewoud> knesenko: I agree, but not sure how it can be established
15:35:24 <knesenko> I am not sure what is the process today :\
15:35:40 <ewoud> traditionally in open source you submit patches and you grow into a maintainer, but with infra that's harder
15:35:53 <knesenko> agree
15:35:59 <Rydekull> Well
15:36:03 <knesenko> but its very important
15:36:10 <quaid> right, we want to be able to give out little tasks people can grow with
15:36:22 <Rydekull> Without any qaccess what so ever, its hard. But you can still be given accounts on machines etc
15:36:33 <Rydekull> and with screen etc, you can in theory, with a mentor, do stuff/be surpervised
15:36:38 <Rydekull> there are tools to collaborate
15:36:39 <ewoud> one problem is trust; just giving root is a bit scary but without it you can't do a lot
15:36:46 <knesenko> ewoud: +1
15:36:50 <quaid> +1 to working out improvements to that page on the ML
15:37:12 <quaid> ewoud: we can figure out where people can help via transparency
15:37:24 <ewoud> I'll kick of a thread
15:37:26 <Rydekull> I think the essential part is to have a mentor being picked out. And that mentor giving tasks, personally
15:37:29 <quaid> i.e. give out sudoless ssh so people can poke around
15:37:40 <dcaro> I also vote for givin 'per user' accounts and give sudo permissions, to have more control than just root pass
15:37:44 <ewoud> #action ewoud start a thread on the ML to start documention the becoming an infra member
15:37:49 <quaid> and offer solutions that an other can be rootnfor
15:37:50 <ewoud> dcaro: +1
15:38:23 <quaid> right, we can create a backup group & get help with that
15:38:27 <knesenko> I think that no one should work as root . only with sudo
15:38:34 <quaid> or log gathering & analysis
15:38:34 <dcaro> maybe we can set up sudo profiles for give access to do some tasks
15:38:42 <quaid> +1
15:38:46 <obasan> +1
15:38:51 <ewoud> dcaro: that implies we have some standard tasks
15:38:52 <quaid> 1. ssh for looking aroun d
15:39:02 <quaid> 2. ssh + limited sudo
15:39:04 <ewoud> would be a good thing to have, but I'm not sure we currently have that
15:39:07 <quaid> 3. full sudo
15:39:11 <dcaro> ewoud: sure
15:39:17 <knesenko> do we have some git with conf. files of the machines or something like that
15:39:17 <knesenko> ?
15:39:29 <quaid> progress 1 to 3
15:39:30 <ewoud> knesenko: not yet, but we aim to set up puppet for that
15:39:36 <eedri_> i think we can start with monitoring jenkins jobs, that can be easilly solved with limited permissions for jenkins ui
15:39:46 <ewoud> quaid: and maybe a 0. attend a meeting / active on ML
15:39:48 <dcaro> eedri_: +1
15:39:54 <knesenko> ewoud: +1
15:40:00 <Rydekull> eedri_: Wouldnt that depend on which part you'd be involved with
15:40:33 <ewoud> let's continue this on the ML
15:40:57 <ewoud> any other business?
15:41:00 <obasan> yes
15:41:12 <obasan> what is the procedure of getting a poweruser access to jenkins?
15:41:26 <obasan> I'd like to add a job that will generate a nightlies of ovirt-live
15:41:31 <eedri_> basically is sending a request to infra list
15:41:44 <eedri_> and getting approval from the list or community
15:41:48 <obasan> I sent such request. who is supposed to authorize that?
15:42:36 <knesenko> +1 fpr obasan
15:42:58 <eedri_> obasan, i don't think your request made it to the list
15:42:59 <quaid> maybe we want to normally use tickets so we don't forget the task?
15:43:00 <Rydekull> obasan: when did you send it?
15:43:10 <Rydekull> obasan: please dig it up from the archives
15:43:13 <obasan> 2/3 days ago.
15:43:25 <knesenko> quaid: +1 for tickets
15:43:41 <knesenko> quaid: do we have some ticket system ?
15:43:53 <Rydekull> https://fedorahosted.org/ovirt/ticket
15:44:16 <obasan> I think it didn't make it. I will resend it.
15:44:50 <knesenko> Rydekull: 10x
15:45:22 <knesenko> ewoud: do you need any help with puppet/foreman installation/configuration ?
15:45:53 <Rydekull> As for other business, short report from the outage last week(which I think should be a item on the agenda)
15:46:06 <Rydekull> We went out for a few hours as you may have seen on the email i sent
15:46:35 <Rydekull> this was due to errors on openshift's side that made our website behave strangely. Only letting through a request once every decade or so
15:46:36 <ewoud> knesenko: I could use some
15:47:06 <clag> is there any technical doc about snapshot ? i search in ovirt wiki but doesn't found. this is not lvm snapshot ,maybe a qcow method but where is put the file ?
15:47:26 <ewoud> sorry, back
15:47:31 <knesenko> ewoud: ok np
15:48:18 <ewoud> knesenko: some time ago I made https://github.com/ekohl/ovirt-infra-puppet as a proof of concept, but it could use some updates
15:48:59 <ewoud> knesenko: I think the first goal should be to create the user accounts so giving access to all our infra is easy
15:49:42 <ewoud> knesenko: and when it's practical we should move the repo to gerrit.ovirt.org
15:49:59 <dcaro> ewoud, knesenko: we can discuss that in the lists maybe? create a list of what we want on puppet and split the tasks?
15:50:07 <ewoud> dcaro: will do
15:50:10 <knesenko> so you want to create users with puppet on all machines ?
15:50:20 <ewoud> #action ewoud re-ignite the puppet discussion on the ML
15:50:46 <ewoud> knesenko: you can define virtual resources in puppet and only create a subset
15:51:06 <ewoud> I think this brings us to the trac review
15:51:35 <ewoud> #topic Trac ticket review
15:51:56 <ewoud> we haven't done this in a while
15:52:34 <ewoud> quaid: could you start with https://fedorahosted.org/ovirt/ticket/5 so we at least have some sort of workflow?
15:53:35 <quaid> heh, yes
15:53:37 <Rydekull> I did close a few tickets last week
15:53:46 <ewoud> Rydekull: I noticed the list was a bit shorter
15:53:52 <quaid> rydekull++ was determined
15:54:07 <Rydekull> I sorted out the ovirtbot, and ConfirmAccount
15:54:10 <quaid> it was nice to close out some items
15:54:17 <Rydekull> and I prodded some others to close some things that had solved themselves
15:54:30 <quaid> ewoud: i'll work #5 today
15:54:45 <ewoud> quaid: cool
15:55:07 <ewoud> https://fedorahosted.org/ovirt/ticket/31 Licensing (rights) link disappears when graphic is not used
15:55:11 <ewoud> I'm not sure what has to be done
15:56:26 <quaid> garrett: ping
15:56:40 <quaid> we have a feew theme items to discuss with garrett
15:56:49 <Rydekull> Yeah
15:57:15 <quaid> I tried to figure out the licensing link, couldn't solve it
15:57:25 <ewoud> quaid: https://fedorahosted.org/ovirt/ticket/27 Obtain entitlements for RHEL for Infrastructure hosts at RackSpace was picked up by you instead of dneary?
15:57:34 <quaid> yes
15:57:47 <quaid> go ahead and reassign plz if not done already
15:57:59 <Rydekull> Is there any specific issue, you'd like me to focus on?
15:58:01 * quaid is on his phone not laptop :)
15:58:26 <ewoud> quaid: done
15:58:35 <ewoud> https://fedorahosted.org/ovirt/report/1 is the report btw
15:58:36 <quaid> i'd like us to share in the Gerrit knowledge
15:59:00 <ewoud> quaid: I'm setting up gerrit at $dayjob and getting some knowledge
15:59:15 <quaid> was Gerrit something Rydekull wanted to learn? /me not clearly recalling
15:59:25 <Rydekull> Yes
15:59:36 <Rydekull> But it sounds as its starting to get crowded also
15:59:52 <quaid> nah, more the merrier, and you did ask :)
15:59:59 <Rydekull> :-)
16:00:19 <quaid> maybe we can setup a one-off meeting with Itamar to learn?
16:00:36 <quaid> eedri: who else admins gerrit?
16:00:57 <Rydekull> Gerrit - gerrit.ovirt.org: iheim, rydekull
16:01:24 <quaid> rydekull: do you want to setup a sprint/meeting with iheim? we can coordinate on infra@
16:02:04 <quaid> all of us who want to learn & help can pick a time, etc.
16:02:06 <Rydekull> Sure, i'll put gerrit as my next task as an apprentice :-)
16:02:09 <dcaro> quaid: I do not administer gerrit, but I could use the knowledge :)
16:02:14 <quaid> +qq
16:02:18 <quaid> +1
16:02:48 <ewoud> #action Rydekull set up a sprint/meeting about gerrit to spread knowledge
16:02:59 <ewoud> I'd like to close
16:03:20 <ewoud> closing in 3
16:03:20 <quaid> +1
16:03:23 <Rydekull> Feel free
16:03:23 <ewoud> 2
16:03:26 <ewoud> 1
16:03:29 <ewoud> thanks all
16:03:30 <ewoud> #endmeeting