15:04:01 <knesenko> #startmeeting oVirt Infra Weekly
15:04:01 <ovirtbot> Meeting started Mon Mar 10 15:04:01 2014 UTC.  The chair is knesenko. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:04:01 <ovirtbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:04:21 <knesenko> #chair bkp dcaro eedri_
15:04:22 <ovirtbot> Current chairs: bkp dcaro eedri_ knesenko
15:06:00 <knesenko> eedri_: want to start with tasks ?
15:06:14 <knesenko> eedri_: dcaro seems like from the infra its only us
15:06:51 <eedri_> knesenko, yea, i think since we're most of the time not getting around to review tasks, it might be better to review tasks for 10-15 at the begining
15:06:54 <dcaro> xd
15:07:03 <dcaro> okko
15:07:05 <eedri_> doron_afk, joining?
15:07:12 <eedri_> Rydekull, ?
15:07:17 <knesenko> #topic Review Tickets/Tasks
15:07:23 <doron_afk> eedri_: sorri but no.
15:07:29 <knesenko> dcaro: eedri_ want to lead this topic /
15:07:30 <knesenko> ?
15:07:40 <eedri_> dneary, around?
15:07:53 <eedri_> knesenko, 1st we need to know how to get admin rights on this trac instance
15:08:02 <eedri_> knesenko, some new fields are needed and updates
15:08:14 <knesenko> eedri_: I think we need to ask dneary quiad for that
15:08:25 <eedri_> bkp, do you have access to trac?
15:08:28 <knesenko> eedri_: I assume one of them created it
15:08:36 <eedri_> bkp, https://fedorahosted.org/ovirt/
15:08:43 * eedri_ looking at the usual place
15:08:54 <ewoud> knesenko: pong
15:08:54 <bkp> eedri_ Let me check...
15:09:01 <knesenko> ewoud: hello ...
15:09:06 <knesenko> ewoud: want to join the meeting ?
15:09:17 <ewoud> knesenko: yes
15:09:49 <knesenko> #chair ewoud
15:09:49 <ovirtbot> Current chairs: bkp dcaro eedri_ ewoud knesenko
15:10:12 <bkp> eedri_ It appears not.
15:10:24 <eedri_> #action add trac admin credentials to infra storage place
15:11:21 <knesenko> ok
15:11:37 <knesenko> eedri_: who will handle that one ?
15:11:58 <eedri_> knesenko, still waiting for dneary to answer on the ticket, i added needinfo on it
15:12:08 <knesenko> eedri_: good ...
15:12:15 <knesenko> eedri_: ok lets start with urgent tickets
15:12:45 <eedri_> knesenko, i'm wondering how should we sort it, not sure the priority field is used
15:13:26 <dcaro> it's a 'little' used
15:14:05 <eedri_> dcaro, can we sort by it?
15:14:25 <dcaro> eedri_: most of the tasks have the same priority there, so you'll get more or less the same
15:15:24 <dneary> eedri_, ? How can I help?
15:15:25 <eedri_> dcaro, ok, i managed to do custom query
15:15:28 <eedri_> dneary, hey!
15:15:41 <eedri_> dneary, i'm hoping you have ovirt trac credentials
15:15:42 <dneary> knesenko, I do not have admin access to the trac instance
15:15:53 <eedri_> dneary, who has then? it's not documented
15:15:53 <dneary> We need to ask the fedorahosted people who can control it I think
15:16:26 <YamakasY> woei!
15:16:26 <eedri_> dneary, any email for support/requests?
15:16:38 <dneary> eedri_, Looking now
15:16:43 <knesenko> eedri_: I think we should try #fedora* on freenode
15:16:53 <dneary> eedri_, I'm asking on #fedora-devel on freenode
15:16:59 <misc> try #fedora-infra, ues
15:17:26 <eedri_> knesenko, http://goo.gl/ARQzFG
15:17:35 <eedri_> knesenko, try this link as sorted by priority
15:18:11 <knesenko> eedri_: ok
15:18:12 <knesenko> https://fedorahosted.org/ovirt/ticket/17
15:18:17 <knesenko> ok this one is for backups
15:18:30 <eedri_> knesenko, yea, we need of course to prioritze this
15:18:44 <knesenko> to be honest I am confused where do we have backups and what do we backup
15:18:50 <eedri_> knesenko, maybe worth shooting email to infra and asking for a volenteer to take ownership on it
15:19:01 <eedri_> knesenko, we need a full mapping of our resources + what is backedup
15:19:08 <knesenko> eedri_: ok
15:19:24 <eedri_> knesenko, right now it's jenkins config backup to alterway, gerrit backup to resources.ovirt.iorg
15:19:24 <knesenko> someone here wants to create a wiki page for this ticket ?
15:19:36 <eedri_> knesenko, so we need to document what we have and what is the gaps
15:19:52 <eedri_> knesenko, i would also change asgineed to infra@ovirt.org
15:20:21 <knesenko> eedri_: please change
15:20:30 <knesenko> eedri_: ok no one here ...
15:20:43 <knesenko> #action eedri_ drop email to infra@ovirt.org regarding - https://fedorahosted.org/ovirt/ticket/17
15:20:45 <knesenko> next
15:20:58 <knesenko> https://fedorahosted.org/ovirt/ticket/111
15:21:00 <dcaro> I'm not sure on changing the asigned to infra, then noone will take it
15:21:18 <dcaro> that's mine :)
15:21:22 <knesenko> dcaro: yes
15:21:30 <knesenko> its not clear for me what is needed from the ticket
15:21:52 <dcaro> we were still discussing the puppet repository structure... and I wanted to have that clear before writing that code
15:22:04 <knesenko> dcaro: related to puppet ?
15:22:31 <dcaro> knesenko: I wanted to add a puppet class to add all that's needed for a slave to run the repoclosure jobs
15:22:54 <knesenko> dcaro: but we do have this code already
15:22:59 <knesenko> dcaro: the job I created ...
15:23:07 <knesenko> dcaro: or I do not follow
15:23:11 <dcaro> knesenko: not the job, but the puppet code related to it
15:23:22 <knesenko> dcaro: ok
15:23:25 <dcaro> knesenko: iirc right now we installed those deps manually
15:23:43 <knesenko> dcaro: can you update the ticket please with the status ....
15:23:46 <knesenko> dcaro: got you
15:23:59 <knesenko> #action dcaro update status https://fedorahosted.org/ovirt/ticket/111
15:24:01 <knesenko> next
15:24:14 <knesenko> https://fedorahosted.org/ovirt/ticket/10
15:24:46 <knesenko> dneary: bkp can you take a look on - https://fedorahosted.org/ovirt/ticket/10
15:24:52 <knesenko> dneary: bkp still relevant ?
15:24:59 <eedri_> knesenko, this one 17 months old :)
15:25:06 <knesenko> eedri_: just asking
15:25:13 <eedri_> knesenko, yea, i know
15:25:17 <dneary> knesenko, pingou is being pointed to as the guy who would know
15:25:20 <eedri_> knesenko, that's why it's good we're reviwing it
15:25:22 <knesenko> eedri_: I don't think its relevant , but still
15:25:23 <bkp> knesenko: Yes, this was something that needed to get done.
15:25:37 <dneary> knesenko, Yes, still relevant
15:25:44 <dneary> But still not top priority
15:25:55 <bkp> ^^^
15:26:09 <ewoud> as time passes, it will become harder and harder to rename
15:26:23 <dneary> If an item on a TODO list is never your top priority, will it ever get done?
15:26:30 <dneary> ewoud, Not really
15:26:40 <dneary> ewoud, The procedure is straightforward
15:26:49 <eedri_> dneary, maybe if it's not done in 17 months then it means its not really needed?
15:26:57 <dneary> Just time consuming & wrought with list downtime & data loss danfger :-)
15:27:17 <ewoud> dneary: let me rephrase: getting people to adapt will become harder and harder
15:27:17 <bkp> eedri_ Or the person it was assigned to was slammed with other things.
15:27:20 <xaviern> ty SvenKieske i missed the answer
15:27:25 <dneary> By the way, shouldn't this meeting have been an hour ago?
15:27:39 <eedri_> dneary, not by israeli time
15:27:40 <bkp> Daylight time?
15:27:43 <eedri_> dneary, its because of dst
15:28:07 <knesenko> ok
15:28:11 <dneary> eedri_, It's in my calendar for 3pm French time every Monday, didn't get moved today
15:28:19 <eedri_> dneary, we should be ok by eom, unless people want to move it one hour back till dst is changed
15:28:19 <dneary> Maybe it's fixed to EST
15:28:22 <knesenko> so what's the decision regarding https://fedorahosted.org/ovirt/ticket/10 ?
15:28:26 <ewoud> dneary: it is
15:28:35 <dneary> knesenko, Leave it open, low priority
15:28:41 <bkp> Can I ping quaid on that and see where he is at with it?
15:28:46 <knesenko> eedri_: can you change its priority please ?
15:28:50 <knesenko> eedri_: I can't login
15:28:54 <bkp> (#10)?
15:29:03 <knesenko> bkp: yep
15:29:08 <ewoud> I'd like to nominate https://fedorahosted.org/ovirt/ticket/83 as the next to discuss
15:29:22 <ewoud> don't we already do this through the CI-tools repo?
15:29:31 <bkp> Changed to minor priority
15:29:32 <sgotliv> jntaylor, ping
15:29:38 <bkp> (I have access now)
15:29:45 <knesenko> ewoud: got it
15:29:45 <eedri_> ewoud, i think so too
15:29:47 <knesenko> https://fedorahosted.org/ovirt/ticket/83
15:29:48 <ewoud> or can we ship a python-nose there with a newer version?
15:30:05 <eedri_> ewoud, didn't see to be an issue anyway lately with it
15:30:16 <eedri_> ewoud, and we're runnign on new slaves, so i guess we're ok, dcaro ?
15:30:20 <knesenko> eedri_: me too
15:30:51 <dcaro> eedri_: I haven't seen the issue lately either
15:31:05 <knesenko> dcaro: eedri_ so I suggest to close it ?
15:31:08 <eedri_> knesenko, dcaro so i think we can close thi
15:31:15 <knesenko> dcaro: eedri_ and reopen if we see it once again
15:31:19 <ewoud> I do see python-nose1.1 is installed from EPEL
15:31:35 <eedri_> ewoud, the problem was if it was installed as default
15:31:41 <eedri_> ewoud, so vdsm will know how to use it
15:31:42 <jntaylor> sgotliv: here
15:32:02 <knesenko> so close it ?
15:32:03 <ewoud> eedri_: ah yes, we don't enforce that so still relevant
15:32:04 <eedri_> ewoud, might have been fixed also by the rpm pkg itseldf
15:32:14 <ewoud> [ekohl@jenkins-slave-vm10 ~]$ rpm -qf /usr/bin/nosetests
15:32:14 <ewoud> python-nose-0.10.4-3.1.el6.noarch
15:32:27 <eedri_> ewoud, ok, so still relevant i assume
15:32:37 <ewoud> [ekohl@jenkins-slave-vm10 ~]$ rpm -qV python-nose | grep bin
15:32:37 <ewoud> ....L....    /usr/bin/nosetests
15:32:37 <ovirtbot> ewoud: Error: "...L...." is not a valid command.
15:33:14 <eedri_> knesenko, i propose last ticket to view today (we're passed our set time already)
15:33:21 <knesenko> eedri_: agree
15:33:29 <eedri_> knesenko, ticket 88 - adding new slaves
15:33:33 <knesenko> eedri_: let me choose the most relevant
15:33:42 <knesenko> ok
15:33:42 <knesenko> https://fedorahosted.org/ovirt/ticket/88
15:33:56 <eedri_> we currently have some slaves offline + are missing some os..
15:34:50 <eedri_> anyone volenteer to take it?
15:35:12 <knesenko> how can we install slaves on rackspace instead of using ISO ?
15:35:18 <knesenko> can we use some network installation ?
15:35:45 <eedri_> ewoud, can we use foreman?
15:36:33 <ewoud> submitted http://gerrit.ovirt.org/25588 for #83
15:36:39 <knesenko> ewoud: dcaro its possible to setup PXE on rackspace network and use it in foreman somehow right ?
15:36:52 <ewoud> knesenko: eedri_ should be possible
15:37:45 <knesenko> dcaro: eedri_ ewoud I assume we can use one of hypervisors as PXE
15:38:40 <dcaro> iirc they are not using internal network, so we should create a local virtual lan for them to enable dhcp there
15:39:09 * ewoud assigned it to himself
15:39:38 <knesenko> dcaro: maybe we should ask them vefore we install DHCP there ... ?
15:39:47 <ewoud> dcaro: if you can review http://gerrit.ovirt.org/25588 we can close #83
15:40:00 <dcaro> ewoud: already looking
15:40:13 <ewoud> knesenko: I have no contact info there, so if any RH employee could take care of that …
15:40:26 <knesenko> ewoud: I can open a ticket to ask them
15:40:38 <knesenko> ewoud: I mean I can ask them via the ticket :)
15:40:46 <ewoud> knesenko: please do
15:41:06 <ewoud> knesenko: say that we do NOT want to add a free lease pool, just the IPs we want to provision
15:41:22 <knesenko> #action knesenko ask rackspace guys if it is possible to install dhcp server for our ip range
15:41:25 <knesenko> ewoud: will do
15:41:28 <knesenko> ok ...
15:41:35 <knesenko> so to summarize
15:41:53 <knesenko> ask rackspace guys about dhcp - and ewoud will setup pxe server there ....
15:42:06 <dcaro> foreman proxy!
15:42:08 <knesenko> if it will take some time - resintall slaves from ISo
15:42:21 <knesenko> sounds good ?
15:42:24 <ewoud> yes
15:42:31 <dcaro> ok
15:42:34 <knesenko> ok
15:42:40 <knesenko> #topic Jenkins
15:42:45 <knesenko> what do we have here ?
15:42:48 <knesenko> eedri_:
15:42:50 <knesenko> dcaro:
15:43:13 <eedri_> knesenko, well
15:43:29 <dcaro> jenkins had a few issues with disk space on the smaller slaves, still working on it
15:43:40 <eedri_> knesenko, i think mostly our need to reinstall/fix rackspace slaves
15:43:50 <eedri_> knesenko, but as dcaro said, mostly space issue
15:44:13 <eedri_> knesenko, there was a request done for allowing access to slaves for power users
15:44:36 <eedri_> knesenko, communicated on the list, and from what i understood quite important to them, as they don't have ability to debug functional tests
15:44:59 <eedri_> any objections to what i proposed on the list?
15:45:20 * eedri_ doesn't see much difference/risk in giving ssh access to slaves to power user who has access to jenkisn master
15:45:35 <eedri_> slaves are in anyhow should be stateless
15:45:55 <eedri_> neverthelss should be managed and controled by infra + puppet class
15:46:00 <dcaro> +1
15:46:34 <knesenko> +1
15:48:11 <eedri_> ewoud, ?
15:49:27 <ewoud> I'd prefer if debugging was not done through SSH
15:49:44 <ewoud> maybe the log files / core dumps / … could be added as artifacts?
15:49:51 <eedri_> ewoud, from what i understood from some developers that was their only way to testing,
15:50:00 <eedri_> ewoud, but i agree archiving the logs could also be done
15:50:04 <eedri_> mskrivanek_away, here?
15:50:24 <eedri_> ewoud, we could try and reach out to the vdsm functional tests owners and ask
15:50:48 <eedri_> ewoud, why ssh access is needed, and see if archving is the right way to go, or ssh access it a must
15:51:20 <eedri_> ewoud, knesenko dcaro can you reply on the thread and share your thoughts on this?
15:51:31 <knesenko> eedri_: ok
15:51:54 <ewoud> eedri_: ok
15:52:33 <eedri_> #agreed cotinue discussion on developers access to jenkins slaves on list
15:52:52 <eedri_> dcaro, fabiand you wanted to discuss improviong ovirt-node jobs?
15:52:54 <knesenko> anything esle on jenkins ?
15:52:58 <eedri_> that they might be taking too much space?
15:53:01 <knesenko> instead of slaves issues we have
15:53:36 <knesenko> ewoud: i asked the m
15:53:43 <knesenko> ewoud: lets see what they say
15:55:24 <eedri_> knesenko, another topic on jenkins is gerrit failures
15:55:44 <knesenko> eedri_: right
15:55:45 <eedri_> knesenko, we changed gerrit trigger jobs to fail only on unstable builds, to reduce blocking patches on false positive infra issues
15:56:09 <eedri_> knesenko, only problem is that now specific failures (real ones) that cause job to be red, like mvn compile error might be missed
15:56:14 <eedri_> knesenko, as it won't give -1
15:56:39 <eedri_> knesenko, so what was proposed is to change these jobs to be unstable instead of failure, to reflect the real nature of the failre
15:56:51 <eedri_> and keep failing red jobs only to unexpected infra issuses
15:57:02 <eedri_> this could be done via groovy code on post build section
15:58:42 <eedri_> dcaro, do we have a trac ticket on this to follow up?
15:59:11 <dcaro> eedri_: nop, opening
16:00:22 <eedri_> knesenko, not related to jenkins, but there was a request to rename nightly build for master
16:00:32 <eedri_> knesenko, i think there was an open ticket on it also
16:00:43 <eedri_> knesenko, well, maybe partically related
16:00:59 <knesenko> eedri_: yes ...
16:01:09 <knesenko> eedri_: i don't know ...
16:01:24 <knesenko> eedri_: we will discuss it with dcaro and sbonazzo later
16:01:30 <knesenko> not urgent for now
16:01:57 * Rydekull is here
16:02:24 <eedri_> knesenko, ok
16:02:39 <eedri_> knesenko, last request was one on infra and i didn't saw any replay
16:02:44 <eedri_> knesenko, was changing a vdsm job
16:02:49 * eedri_ looking for details
16:02:57 <knesenko> eedri_: ok
16:03:29 <eedri_> knesenko, "can you replace in vdsm upstream build the "make" command in "make all"?"
16:03:34 <eedri_> knesenko, sent on the list yesterday
16:03:38 <eedri_> anyone picked that up?
16:03:46 <eedri_> dcaro, ?
16:03:53 <dcaro> eedri_:
16:04:26 <eedri_> imo this is something that can be also done by a vdsm power user, just not sure who that is
16:04:58 <knesenko> eedri_: i changed the vdsm buiukd
16:05:10 <knesenko> eedri_: there is no make there
16:05:14 <knesenko> eedri_: now the flow id
16:05:15 <knesenko> s
16:05:29 <knesenko> autoreconf -avf; ./configure; make dist
16:05:37 <knesenko> and rpmbuild ... blah blah
16:05:53 <eedri_> knesenko, can you reply to ybronheim then?
16:06:04 <knesenko> eedri_: email ?
16:06:10 <eedri_> knesenko, yea, email sent to infra
16:06:14 <knesenko> eedri_: ok
16:06:16 <dcaro> eedri_: depends on the jenkins job
16:06:20 <knesenko> guys we are our of time
16:06:28 <eedri_> dcaro, knesenko yea, he should provide more details
16:06:42 <eedri_> knesenko, ok, we'll continue next time, unless someone has an urgent issue we didn't cover?
16:06:56 <knesenko> eedri_: corect
16:07:11 <knesenko> thank you guys !
16:07:16 <knesenko> have a nice week
16:07:37 <knesenko> ewoud: thanks for the the foreman proxy
16:07:39 <knesenko> :)
16:07:41 <knesenko> #endmeeting