14:03:58 <ewoud> #startmeeting infra weekly meeting
14:03:58 <ovirtbot> Meeting started Mon Jun 17 14:03:58 2013 UTC.  The chair is ewoud. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:03:58 <ovirtbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:04:13 <ewoud> agenda:
14:04:14 <ewoud> Introductions
14:04:14 <ewoud> Review of action items
14:04:14 <ewoud> Hosting
14:04:17 <ewoud> Puppet
14:04:17 * knesenko here
14:04:19 <ewoud> Jenkins
14:04:22 <ewoud> Other business?
14:04:22 * obasan here
14:04:24 <ewoud> Trac review
14:04:35 <knesenko> eedri: meeting
14:04:40 <ewoud> #chair obasan knesenko
14:04:40 <ovirtbot> Current chairs: ewoud knesenko obasan
14:05:35 <ewoud> #topic hosting
14:06:17 <ewoud> knesenko: I saw your mail and I was wondering if we need all in one for local storage
14:06:23 <juliano> knesenko, http://resources.ovirt.org/releases/3.2/rpm/EL/6/noarch/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
14:06:57 <obasan> juliano, put this http://resources.ovirt.org/releases/3.2/rpm/EL/6/
14:07:17 <obasan> juliano, do not include arch in the repo url
14:07:27 <knesenko> ewoud: yes ... one all-in-one setup
14:07:38 <knesenko> ewoud: will create 2 DCs in it
14:07:42 <knesenko> 2 local DCs
14:07:45 <knesenko> for jenkins slaves
14:07:56 <knesenko> as we discussed on mailing list
14:08:05 <ewoud> juliano: I'd recommend installing http://resources.ovirt.org/releases/ovirt-release-el.noarch.rpm instead of manually writing the repo files
14:08:16 <knesenko> ewoud: +1 :)
14:08:17 <ewoud> knesenko: ok, but it will be one ovirt instance?
14:08:30 <knesenko> ewoud: yes ... i sent a new diagram
14:09:17 <knesenko> ewoud: sounds ok ?
14:09:57 <ewoud> knesenko: I'm looking at http://lists.ovirt.org/pipermail/infra/2013-June/003301.html
14:10:49 <ewoud> knesenko: but we will have one engine, do I see that right?
14:11:45 <knesenko> ewoud: yes one engine
14:11:55 <knesenko> ewoud: one sec looking for the correct email
14:12:25 <knesenko> ewoud: weird I can't find it here - http://lists.ovirt.org/pipermail/infra/2013-June/
14:12:25 <ewoud> knesenko: sounds good
14:12:50 <ewoud> knesenko: it's not the one I linked a few minutes ago? because that's the only one I received
14:13:13 <knesenko> ewoud: I am sorry ... I sent to eedri_ and not to the infra@ ...
14:13:18 * knesenko sending
14:14:23 <obasan> knesenko, http://en.wikipedia.org/wiki/Netsplit
14:14:35 <knesenko_> ewoud: take a look on the mailing list now
14:15:21 * eedri_ has issues with irc, online now
14:15:55 <juliano> knesenko_, it still error Error: Package: vdsm-gluster-4.10.3-17.el6.noarch (ovirt-centos) Requires: glusterfs >= 3.4.0
14:16:20 <juliano> using this repo this error http://ovirt.org/releases/stable/rpm/EL/6Server/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
14:16:24 <eedri_> ewoud, knesenko we're good with AIO for rackspace?
14:16:28 <ewoud> #chair eedri_
14:16:28 <ovirtbot> Current chairs: eedri_ ewoud knesenko obasan
14:16:38 <eedri_> ewoud, knesenko or we prefer having a seperate VM to run engine?
14:16:57 * eedri_ wondering how hard it will be to migrate AIO to a seperate engine if needed in the future
14:17:05 <obasan> #chair knesenko
14:17:05 <ovirtbot> Current chairs: eedri_ ewoud knesenko obasan
14:17:06 <ewoud> eedri_: I don't see the new mail knesenko_ sent yet so I can't tell
14:17:16 <obasan> #chair knesenko_
14:17:16 <ovirtbot> Current chairs: eedri_ ewoud knesenko knesenko_ obasan
14:17:29 <knesenko_> ewoud: eedri_ http://lists.ovirt.org/pipermail/infra/attachments/20130617/8063fc2e/attachment-0001.pdf
14:18:01 <ewoud> knesenko_: where will that all-in-one run?
14:18:14 <knesenko_> ewoud: eedri_ no need for a separate engine in rackspace ... we will use all-in-one with 2 local storage DCs
14:18:38 <knesenko_> ewoud: will choose ... rackspace01-02 ... I don't care
14:18:49 <eedri_> knesenko, ok, if that suffice i vote to push it forward asap
14:18:55 <knesenko_> ewoud: same specs right ?
14:18:56 <eedri_> knesenko, nothing is blocking us , hosts are ready
14:19:06 <knesenko_> eedri_: ping knesenko_
14:19:25 <eedri_> knesenko_, k
14:19:30 * dcaro is back
14:19:39 <ewoud> knesenko_: I think one is a bit beefier than the other in terms of RAM and storage
14:19:50 <ewoud> I vote we use that one as engine :)
14:19:57 <ewoud> and +1 on pushing it forward
14:20:03 <knesenko_> ewoud: ok
14:20:18 <knesenko_> so rackspace01 is the engine ?
14:20:21 <knesenko_> eedri_: obasan ?
14:20:28 <ewoud> knesenko_: if that one is the stronger one, yet
14:20:30 <obasan> knesenko, +1
14:20:32 <eedri_> knesenko_, if you need info on the hosts me and dcaro has access to rackspace website
14:20:46 <eedri_> knesenko_, which holds informantion on the hosts
14:21:00 <eedri_> ewoud, we're going with stable ovirt 3.2.2?
14:21:03 <ewoud> do the rackspace machines have a public IP address?
14:21:08 <eedri_> ewoud, yes
14:21:12 <knesenko_> eedri_: no need ... I can ssh and check this info
14:21:30 <ewoud> eedri_: I prefer the latest ovirt stable yes
14:21:42 <eedri_> ewoud, rackspace01.ovirt.org
14:21:48 <eedri_> ewoud, rackspace02.ovirt.org
14:21:52 <ewoud> bit of connection issues the past week for the RH nat-pool? seeing lots of disconnects
14:22:00 <eedri_> ewoud, firewall rules are set also
14:22:33 <ewoud> eedri_: I see, I can't connect :)
14:22:33 <knesenko> ewoud: ok .. so I will install it this week
14:22:47 <eedri_> ewoud, ssh?
14:22:49 <ewoud> #action knesenko install rackspace servers as discussed on the ML
14:22:52 <ewoud> eedri_: es
14:23:09 <eedri_> ewoud, right
14:23:18 <eedri_> ewoud, they set it from jenkins.ovirt.org
14:23:23 <knesenko> ewoud: I extended the storage on alterway seetup
14:23:25 <eedri_> ewoud, so access is only possible from there
14:23:27 <ewoud> knesenko: cool
14:23:28 <ewoud> eedri_: ok
14:23:31 <eedri_> ewoud, unless you connect the the vpn
14:23:35 <eedri_> knesenko, ^^
14:23:54 <knesenko> eedri_: what ? :)
14:24:09 <eedri_> knesenko, acccess to rackspace server is from jenkins.ovirt.org
14:24:17 <knesenko> eedri_: ok
14:24:30 <ewoud> anything else on hosting?
14:24:39 <knesenko> juliano: i don;t get your problem .... which error are you getting ?
14:24:41 <knesenko> 1.  http://ovirt.org/releases/stable/rpm/EL/6Server/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
14:24:50 <knesenko> 2. error Error: Package: vdsm-gluster-4.10.3-17.el6.noarch (ovirt-centos) Requires: glusterfs >= 3.4.0
14:24:50 <eedri_> ewoud, do we want to migrate services from linode to alterway?
14:24:55 <knesenko> juliano: which one ?
14:24:59 <eedri_> ewoud, now that we have more space there
14:25:37 <ewoud> knesenko: sounds like you're running RHEL and there are only resources for CentOS
14:26:10 <ewoud> eedri_: sounds good
14:26:13 <knesenko> ewoud: that's not me ... I am trying to help juliano
14:26:15 <ewoud> resources first?
14:26:17 <obasan> knesenko, ewoud I installed successfully from this repository on centos
14:27:03 <knesenko> ewoud: nm ... lets move on
14:27:29 <eedri_> ewoud, yes
14:27:40 <eedri_> ewoud, i suggest to migrate the resources.ovirt.org 1st
14:28:05 <eedri_> ewoud, should be a simple rsync and updating jenkins publish job
14:28:13 <ewoud> eedri_: I'd like to fully puppetize the machine first tbh
14:28:17 <eedri_> ewoud, ok
14:28:18 <knesenko> i would like to see a migration plan for this
14:28:24 <ewoud> should be rather easy
14:28:49 <eedri_> knesenko, obasan can you take ownership on this migration?
14:29:03 <knesenko> eedri_: yes AI am
14:29:18 <eedri_> knesenko, great
14:29:29 <eedri_> knesenko, so let's do it after the rackspaec install
14:29:38 <eedri_> knesenko, and write up a plan for it
14:29:53 <knesenko> eedri_: of course ... no migration without a plan
14:30:08 <knesenko> eedri_: will send it to infra@
14:30:12 <eedri_> knesenko, great
14:30:17 <eedri_> ewoud, actions?
14:30:27 <Yamaksi> was it possible to boot CentOS/Ovirt nodes over PXE ? I remember something about the docs that needed to be updated about it.
14:31:07 <obasan> #action knesenko create migration plan for linode resources to alterway
14:31:11 <knesenko> good
14:31:14 <knesenko> next
14:31:33 <juliano> knesenko, im really trying to install on RHEL now it return this error 2013-06-17 10:04:19 ERROR otopi.context context._executeMethod:139 Failed to execute stage 'Package installation': No package(s) available to instal
14:31:35 <eedri_> #action knesenko to install AIO setup on rackspace01 and start adding jenkins slaves vms
14:31:54 <juliano> 2013-06-17 10:04:19 ERROR otopi.plugins.otopi.packagers.yumpackager miniyumlocal.error:58 Yum Cannot queue package tuned: No package(s) available to install
14:32:25 <juliano> knesenko, vdsm is installed why this error?
14:32:43 <eedri_> juliano, can you wait 30 min till we finish the infra meeting?
14:32:58 <eedri_> juliano, i'm sure we can assist you once we're done
14:33:14 <knesenko> juliano: You have a problems with your repos . please recheck them
14:33:24 <juliano> sure, im sorry about it
14:33:39 <ewoud> ok, anything else about hosting?
14:33:49 <eedri_> ewoud, i would like to see monitoring stats
14:33:53 <eedri_> ewoud, like dcaro added on gerrit..
14:33:59 <eedri_> ewoud, could be very usefull
14:34:00 <ewoud> eedri_: +1
14:34:08 <knesenko> ewoud: I would like to push https://fedorahosted.org/ovirt/ticket/49
14:34:09 <obasan> eedri_, +1
14:34:23 <eedri_> dcaro, can you share your ideas, tools for adding stats tot gerrit?
14:34:24 <ewoud> personally I like munin and quite easy to install, also using puppet exported resources
14:34:35 <eedri_> dcaro, it was very useful to track down source load issues
14:35:23 <dcaro> eedri_: ewoud I'used to graphite, that let's you create graphs on the fly, is really easy troubleshoot problems that way
14:35:25 <eedri_> btw, maybe on rackspace it's built in with the service.. we should check that
14:35:48 <eedri_> dcaro, can we setup something permament on monitoring.ovirt.org?
14:36:02 <eedri_> dcaro, on a different port than the nagios server
14:36:09 <ewoud> dcaro: I did see the output you mailed, but how easy can we add our infra using puppet?
14:36:18 <dcaro> eedri_: sure
14:36:25 <ewoud> #chair dcaro
14:36:25 <ovirtbot> Current chairs: dcaro eedri_ ewoud knesenko knesenko_ obasan
14:37:04 <eedri_> dcaro, i think we can benefit from it... worth adding as an action item for next week if possible
14:37:26 <eedri_> dcaro, or sending info to other infra members on how to do it
14:37:49 <dcaro> ewoud: there are already some classes to manage graphite, it's formed of one service that receives the data, and one that showsh the info (actually there are some frontends)
14:38:17 <dcaro> eedri_: ok, I'll try to set it up
14:38:36 <ewoud> #action dcaro set up graphite as trending tool
14:38:58 <ewoud> more on hosting?
14:39:17 <ewoud> I think we can put the wiki under hosting and I don't like we ran out of disk space again
14:40:06 <eedri_> ewoud, you mean migrate our of openshift?
14:40:28 <ewoud> eedri_: as in the agenda
14:40:44 <ewoud> how are we going to monitor our openshift quota?
14:40:53 <ewoud> and we should fix the logging issue
14:40:55 <eedri_> ewoud, right..
14:41:26 <obasan> ewoud, eedri_ there is openshift cli/api
14:41:48 <obasan> ewoud, eedri_ but I don't know what kind of info it can provide
14:42:17 <eedri_> obasan, can you spend some time and see if they provide monitoring capabilities?
14:42:28 <obasan> eedri_, yes
14:42:36 <eedri_> obasan, +1
14:42:46 <ewoud> #action obasan look into monitoring openshift quota
14:42:50 <eedri_> obasan, i would also look online for monitoring solutions on openshift
14:42:59 <obasan> eedri_, that is what I intend to do
14:44:57 <knesenko> ewoud: ok lets go on
14:44:59 <eedri_> ewoud, ok let's continue?
14:45:04 <ewoud> ok
14:45:08 <ewoud> #topic puppet
14:45:31 <knesenko> yeah what about it ? do we have a good new s here :) ?
14:45:31 <ewoud> we should do more
14:46:05 <knesenko> dcaro: any progress ?
14:46:05 <ewoud> it's all quite basic and I'd like to grow
14:46:26 <ewoud> as I said, I think we should puppetize the new resources.ovirt.org and then migrate everything
14:46:32 <dcaro> knesenko: nop, did not have the time :S
14:46:53 <knesenko> dcaro: ok :(
14:47:07 <knesenko> ewoud: do we have a user class and ssh class ?
14:47:17 <ewoud> knesenko: can you also add rackspace0{1,2} as puppet clients?
14:47:28 <ewoud> knesenko: yes and I already use them to install my user and keys
14:47:30 <knesenko> ewoud: to distribute ssh keys and users over our server ?
14:47:31 <knesenko> s
14:47:46 <knesenko> ewoud: ok
14:47:55 <knesenko> ewoud: I will add rackspace servers
14:48:12 <ewoud> while we wait for official hosting, it's still on https://github.com/ekohl/ovirt-infra-puppet
14:48:27 <ewoud> knesenko: do fork and PR if you want me to add a user for you
14:49:14 <dcaro> knesenko: I started installing puppet, but had some issues, is still pending, ping me when you start with it if you need help
14:49:34 <ewoud> dcaro: could that be firewall related?
14:49:58 <dcaro> ewoud: nono, it was about ruby version
14:50:00 <dcaro> and gems
14:50:08 <knesenko> dcaro: ewoud ok
14:50:09 <ewoud> dcaro: ah
14:50:56 <ewoud> I also see that itamar hasn't made the repo yet
14:51:24 <ewoud> #action ewoud ensure the puppet repo on gerrit.ovirt.org is created
14:51:31 <ewoud> ok, jenkins
14:51:33 <ewoud> #topic jenkins
14:51:36 <ewoud> anything to report?
14:51:38 <Yamaksi> ewoud: are you doing a lot of traffic on the Centos mirror ?
14:51:46 <knesenko> ewoud: try to ping him maybe ?
14:52:05 <ewoud> Yamaksi: not sure, we need to set up some trending tools to get those stats :)
14:52:15 <ewoud> knesenko: I will, I just realised it hasn't been created yet
14:52:17 <Yamaksi> #topic ewoud is changing topics all the time!
14:52:38 <ewoud> eedri_: dcaro anything new with jenkins?
14:52:49 <Yamaksi> ewoud: I wouldn't be a mirror I think
14:53:03 <eedri_> ewoud, well.. i would really like to see githum mirroring back for ovirt-engine
14:53:23 <eedri_> ewoud, i think jenkins will perform better clonening from it than from gerrit.ovirt.org
14:53:29 <dcaro> eedri_: that's on me
14:53:38 <eedri_> ewoud, and it would reduce load from it
14:53:58 <ewoud> eedri_: do note that gerrit has a small delay in mirroring so I'm not sure if gerrit triggers won't fire too fast
14:54:14 <eedri_> ewoud, it's ok, 95% of the jobs are post merge
14:54:27 <eedri_> ewoud, they can keep clone from gerrit.ovirt.org
14:54:34 <eedri_> ewoud, the 5% i mean (per-patch jobs)
14:54:43 <ewoud> Yamaksi: when we have more control over the environment, we're going to look in more mirroring
14:54:49 <ewoud> eedri_: ok
14:55:49 <ewoud> anything else on jenkins?
14:56:47 <eedri_> ewoud, i noticed f17 slave was down
14:57:08 <ewoud> eedri_: I did notice it was also quite overloaded the past few days
14:57:35 <Yamaksi> ewoud: hehe, so you can bring back your bandwidth :P
14:58:15 <dcaro> ewoud: right now we have a lot (around 50) find_bugs jobs in queue...
14:58:18 <ewoud> Yamaksi: I'd hope to provide higher speeds rather than use less bandwidth :P
14:58:29 <itamar> ewould - what was the final name for the puppet repo?
14:58:40 <Yamaksi> ewoud: if that is the case there is already something terrible wrong!
14:59:08 <ewoud> itamar: I think infra-puppet would be the best if you want something other than just puppet
14:59:29 <ewoud> Yamaksi: last time I downloaded I only got 4 or 5 MB/s which I think is too little
14:59:34 * eedri is back
14:59:44 <Yamaksi> ewoud: on what kind of line ?
14:59:51 <ewoud> Yamaksi: 100mbit/s
15:00:07 <ewoud> eedri: did you get that part about it being overloaded?
15:00:11 <eedri> ewoud, no
15:00:17 <Yamaksi> ewoud: I thought you guys had 10GB lines ?
15:00:20 <eedri> ewoud, irc threw me out
15:00:22 <ewoud> ok, I did notice that so maybe that was related
15:00:24 <ewoud> Yamaksi: it was at home
15:01:01 <ewoud> eedri: also, maybe it's time to upgrade the f17 slave to f19?
15:01:15 <ewoud> or do we just wait for the new rackspace slaves?
15:01:26 <eedri> ewoud, i think so yea.. if we don't need to support f17 rpms anymore
15:01:35 <ewoud> sounds good
15:02:05 <itamar> ewould - who are maintainers for infra-puppet (should be a limited set of folks for +2/merge)
15:02:44 <ewoud> itamar: I think initially me and dcaro
15:02:50 <eedri> itamar, i propose ewoud dcaro
15:03:04 <itamar> ok. done
15:03:12 <Yamaksi> ewoud: what has that something todo with your work enviroment ?
15:03:26 <itamar> ewould --^ git/group/maintainers - done
15:03:26 <Yamaksi> ewoud: I wonder if that 100mbit/s is guaranteed
15:03:35 <ewoud> itamar: thanks
15:03:42 <dcaro> itamar: thanks!
15:03:49 <ewoud> #topic other business
15:03:54 <ewoud> anything else to discuss?
15:05:33 <dcaro> wee need the mailing list for bugzilla/gerrit hooks
15:06:41 <ewoud> dcaro: ?
15:06:54 <ewoud> not quite sure what you mean by that
15:07:54 <dcaro> ewoud: the hooks need an email account to write comments and modify bugs in bugzilla, and I asked for a new mailing list for that
15:08:10 <ewoud> dcaro: ah
15:08:21 <ewoud> does it need to be a ML or just an account?
15:08:46 <ewoud> and can we manage that more generic maybe? relate it the linkedin email maybe?
15:09:52 <dcaro> actually, an account is enough, but I though that the mailing list would make it easier to manage (all the people that has to mantain the hooks/bugs, on the list for example)
15:10:02 <ewoud> true
15:10:23 <Yamaksi> ewoud: why are you changing the topic every time
15:10:52 <dcaro> maybe using the same as the linkedin profile can end in mails not interesting to anyone in the list
15:11:05 <Yamaksi> #topic who follows ewoud ??
15:11:16 <ewoud> Yamaksi: meeting, this way it shows up in the minutes as topic
15:11:34 <Yamaksi> ewoud: okay
15:11:45 <ewoud> dcaro: do you want the ML for BZ changes to be public?
15:12:18 <dcaro> ewoud: no, because that can lead to someone changing the pass or using it to modify bugs in it's name
15:13:15 <ewoud> dcaro: ok, given the time let's continue this on the ML
15:13:21 <dcaro> ewoud: o
15:13:25 <dcaro> *ok
15:14:15 <ewoud> I'd like to end the meeting so if there's nothing else
15:14:52 <ewoud> going once
15:15:20 <ewoud> going twice
15:15:47 <ewoud> thanks all
15:15:48 <ewoud> #endmeeting