14:20:37 #startmeeting 14:20:37 Meeting started Mon Oct 27 14:20:37 2014 UTC. The chair is dcaro. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:20:37 Useful Commands: #action #agreed #help #info #idea #link #topic. 14:20:49 #chair eedri bkorren bkp sbonazzo 14:20:49 Current chairs: bkorren bkp dcaro eedri sbonazzo 14:21:23 #topic Hosting 14:21:42 #info Moving away from rackspace 14:22:07 I'm glad to announce that we are ready to decommission the rackspace servers 14:22:11 \0/ 14:22:12 whooho 14:22:19 \0/ 14:22:26 +1 .. on a great effort finally finished 14:22:31 dcaro, ^ 14:22:39 great :-) 14:22:41 they have been out already almost a week, we moved all the servers from there to phoenix lab 14:22:43 dcaro, that's for the effort and leading this project 14:22:50 welcome :) 14:22:54 thanks also for bkorren in asissting 14:23:26 ofrenkel: indeed! 14:23:42 ofrenkel: BUT my centos 6 box is a 3.4 host 14:23:45 hopefully we can start stabilizing the envs to get reliable builds 14:23:50 eedri: brought cookies ? 14:24:05 YamakasY, brownies 14:24:13 eedri: ++++++++++! 14:26:15 eedri, neah, credit is all for dcaro I just got in his way ^^;; 14:26:52 bkorren: don't underestimate your getting in the way ;) 14:28:06 this sucks :( 14:28:16 stumbling from one bz to another 14:28:23 dcaro, oh, btw, I've seen some stuff fail because of missing postgres, is the intention to have it installed everywhere or was it a badly labeled slave? 14:28:41 bkorren: I'm writing right now a patch to get it everywhere 14:30:00 dcaro are we moving to topic "jenkins slaves issues"? :-) 14:30:05 dcaro, via puppet? you might like to try puppetlabs` postgresql module - its reasonably good, and can save you time... 14:30:07 #topic jenkins 14:30:39 bkorren: I will :) thanks 14:31:01 #info Add postgresql confs tot he slaves, look into puppetlabs postgresql module 14:31:21 sbonazzo: feel free to expose issues :) 14:32:05 dcaro: we have vdsm-jsonrpc-java failing the build for F19 on F21 slaves, I guess it's something different than the rpm bug 14:32:20 dcaro looks something more network / jenkins related 14:33:11 dcaro http://jenkins.ovirt.org/job/vdsm-jsonrpc-java_master_create-rpms-fc19-x86_64_merged/71/console 14:33:15 sbonazzo: I just reconnected one of the slaves, that seemed to be online but failing to ping (not sure why jenkins still got it online) 14:33:23 dcaro oh! 14:33:34 dcaro ok, now we have the rpm bug on f21 :-) 14:33:41 sbonazzo: :/ 14:34:16 sbonazzo: yep, I'll add the vdsm-jsonrpc labef filter too, I suppose that it will happen on all the fc19 jobs that run on fc21, I'll review all 14:34:20 dcaro but for vdsm-jsorpc-java on master we can drop f19 build 14:34:27 dcaro a 3.5 branch has been created 14:34:30 sbonazzo: that's better :) 14:34:35 dcaro we need new jobs for that branch 14:34:46 sbonazzo: so those are 2 different issues 14:34:52 dcaro yep 14:35:19 #action dcaro avoid running fc19 mock builds on fc21 hosts 14:35:41 #action dcaro add new 3.5 branch jobs for vdsm-jsonrpc-java project 14:35:49 dcaro +1 14:36:05 sbonazzo: do you know if there's already a ticket for the new jobs? 14:36:36 dcaro I'm not sure, pkliczew, did you open a ticket? 14:36:58 sbonazzo, I forgot to open one, sorry 14:37:06 dcaro so, no, no ticket 14:37:21 pkliczew: can you open it please? 14:37:31 No brownies for pkliczew :) 14:37:43 dcaro, sure 14:37:55 pkliczew: thanks (: 14:39:27 dcaro lat issue on my side 14:39:42 dcaro http://jenkins.ovirt.org/job/ovirt-appliance_engine-3.5_master_merged/ is taking 3 days to finish 14:39:49 dcaro I guess it's stuck 14:40:05 16:43:55 What do you want to do now? 16:43:55 1) Report Bug 16:43:55 2) Debug 16:43:55 3) Quit 16:43:55 14:40:06 sbonazzo: it's waiting user input :/ 14:40:11 16:43:55 What do you want to do now? 14:40:13 16:43:55 1) Report Bug 14:40:15 16:43:55 2) Debug 14:40:17 16:43:55 3) Quit 14:40:48 dcaro yep, we need something to stop the job after some time of inactivity 14:41:12 dcaro (and maybe fix the job) 14:41:24 I think anaconda it's not handling correctly some exception, and generating one that hides the original one 14:41:41 dcaro so anaconda bug? 14:41:58 #action add a timeout to the job ovirt-appliance_engine-3.5_master_merged 14:42:07 fabiand: ^ 14:42:12 fromani: have you seen that error? 14:42:25 dcaro, yes 14:42:33 I actually worked on that already, it depends on a fix for anaconda 14:42:36 let me dig out the fix 14:43:00 dcaro: I don't 14:43:08 https://bugzilla.redhat.com/show_bug.cgi?id=1155979 14:43:08 https://bugzilla.redhat.com/show_bug.cgi?id=1155981 14:43:09 dcaro, ^^ 14:43:19 * eedri here again 14:43:21 fromani: oops xd, that was for fabian 14:43:25 dcaro, these fixes are also for some other jobs which hang. . 14:43:26 strange that he answered... 14:43:46 fabiand, dcaro do we have node nightly builds? someone asked me today 14:43:54 sorry if offtopic 14:43:57 dcaro, and IO think it's a bug in jenkins that it sometimes does not kill stuck jobs .. 14:44:19 eedri, no not currently afaik 14:45:12 dcaro: :) 14:45:59 fabiand: The timeout plugin was updated the last time we upgraded jenkins, and it seems it's not totally backwards compatible, as I just added an action to take (abort build). 14:46:18 fabiand: when was the first time you experienced those timeout errors? 14:47:02 eedri: http://resources.ovirt.org/pub/ovirt-master-snapshot/iso/ovirt-node-base-iso/ <- ? 14:47:43 dcaro, it's actually a bug in anaconda, I experienced it ~1week ago .. 14:47:58 dcaro, the timeout plugin never worked for those jobs .. 14:48:03 not that I can recall .. 14:48:17 fabiand: mmm, ok, I'll investigate 14:48:22 thanks 14:48:24 * fabiand too :) 14:48:29 but how to fix the stalling ;) 14:48:36 #action Investigate timeout plugin not working on node jobs (and possibly others) 14:48:53 fabiand: what is required for anaconda to work properly? 14:48:58 dcaro, timeout is actually configured for all the jobs that currently stall .. 14:49:07 dcaro, an anaconda fix, which landed 14:49:19 it will soon be ion one of their builds which the job will pick up 14:51:07 fabiand: ok, so can we just wait? Or must we work around it untill it's built? 14:51:49 I'll wait 14:51:58 I'll consider to disable the jobs if it will take longer 14:52:20 fabiand: okok 14:52:20 but the fix is merged, so I expect it to land soon (that is what the anaconda team told me) 14:52:32 but let me know if you want me to do sth with those jobs .. 14:54:39 rbarry, tlitovsk fabiand -- friendly reminder to update the team status 14:54:41 on the pad 14:54:43 http://cloudinf.etherpad.corp.redhat.com/RHEVHMinutes? 14:54:54 fabiand: I'll just readjusted the timeout settings there, let me run a couple builds to make sure it works as expected 14:55:07 dcaro, cool! 14:55:09 okok, any other jenkins issues around? 14:55:15 I'll be happy to see ifthat it works! 14:55:19 dcaro, otherwise I am good 14:55:30 Now that I've learned that I got a username+password .. 14:55:31 :) 14:56:35 dcaro, which job will you actually be trigegring? 14:56:41 fabiand: hehehe, i actually was thinking on add the option to the timeout plugin to reset the time when output is generated (for example if it's having network issues and has to retry a few yum mirrors), if the issue persists I'll dig in to fix it too 14:56:51 fabiand: http://jenkins.ovirt.org/job/ovirt-appliance_engine-3.5_master_merged/ 14:57:00 fabiand: did not trigger it as it already has a pending build 14:57:10 fabiand: but reconfigured it 15:00:27 moving 15:00:36 #topic puppet & foreman 15:00:58 #Sent a patch to move the jenkins_slave related manifests to it's owwn module 15:01:04 oops... 15:01:08 #info Sent a patch to move the jenkins_slave related manifests to it's owwn module 15:01:33 dcaro, great, this leads me to question (not about the patch) 15:02:17 dcaro, we have r10k there right? any docs on how it works? can we use feature branches in puppet? 15:02:45 bkorren: every puppet branch is translated into a puppet environment by r10k 15:03:12 bkorren: we have a basic setup, I think there are no docs, we are using only the production branch, but all of them are created on foreman, so they will be deployed to the puppetmaster if created 15:03:40 bkorren: we don't follow any feature-branch flow actually as there's not too much development around them either and we did not agree on a common structure 15:04:16 dcaro, hmm... is it even possible to use feature branches with gerrit? 15:04:46 bkorren: yes you can, but there are no pull requests (if that'ss where you want to go) 15:05:26 bkorren: yes, you can create a review against a branch 15:05:47 it will still need to be merged, but in theory you could do that yourself 15:05:53 if you want it in practice is another 15:06:15 dcaro, I don't mind rebasing to the dev branch when the code is ready, but will gerrit actually let me push arbitrarily named branches? 15:06:39 bkorren: it's not configured to allow it right now, but you can configure it to allow that 15:08:18 bkorren: we can discuss that in the mailing list. If we are going to adopt a differwnt workflow thatn the rest of projects (and force a different gerrit configuration) I'd like to be sure it's better than the alternative 15:09:39 dcaro, ok, I think puppet is special in this case because you need a master to do interesting stuff, but I'll take it to the mailing list when it'll be more relevant 15:11:18 bkorren: ok, good, I'll be glad to discuss it :) 15:11:52 okok, we are almost out of time, any other issues that I forgot about? 15:12:04 dcaro, the wiki :) 15:12:52 bkorren: well, the main promoters to move the wiki are not around, want to talk about it anyhow? 15:13:35 dcaro, I just wanted to remind bkp to write some of what he told us this morning 15:13:51 bkorren: xd 15:13:55 e.g. in the mailing list 15:16:13 bkorren: It's on my list! 15:16:22 bkp, thanks! 15:16:28 Granted, my list is 5 km long... 15:16:44 I forgot! 15:17:27 due to some infra members being on the other side of the atlantic, we might want to change the meeting time to a later hour 15:18:02 I'll send an email to the list so everyone can comment, but for the people here, what do you think? 15:18:23 (The idea was to move it to ~6:00pm CEST) 15:19:48 Better? Worse? ideas? 15:20:09 dcaro, seems reasonable 15:20:16 dcaro, to me 15:21:54 dcaro, no prob from my side 15:22:07 ewoud, YamakasY Rydekull_ misc ? 15:22:28 * ewoud reads 15:22:56 since that would move it outside of office hours for me, that would be better to me 15:23:45 okok, out of time, i'll start the discussion on the list 15:24:01 so +1 from my side 15:26:09 #endmeeting