14:01:01 #startmeeting 14:01:01 Meeting started Tue Jul 24 14:01:01 2012 UTC. The chair is quaid. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:01 Useful Commands: #action #agreed #help #info #idea #link #topic. 14:01:10 #meetingname oVirt Infrastructure weekly meeting 14:01:10 The meeting name has been set to 'ovirt_infrastructure_weekly_meeting' 14:01:20 #topic roll call & agenda check 14:01:38 * mburns here 14:01:59 * quaid is here 14:02:09 14[[07Category:Infrastructure14]]4 !10 02http://wiki.ovirt.org/w/index.php?diff=3951&oldid=3889&rcid=4047 5* 03Ekohl 5* (+0) 10Correct user page link 14:02:14 * ewoud here 14:02:22 ewoud, i added loop to ignorel ist 14:02:31 eedri: cool 14:02:52 http://wiki.ovirt.org/wiki/Infrastructure_team_meetings#2012-07-24 14:03:15 Agenda 14:03:17 Repo layout for the upcoming 3.1 release - rmiddle proposal Jenkins backups and code review - eedri Jenkins drive space issue. Jenkins migration from EC2 - status Push on ovirt-engine RPMs sync to ovirt.org - status Enabling Gerrit patches - everyone vs. limited All other business 14:03:51 * Repo layout for the upcoming 3.1 release - rmiddle proposal 14:03:51 * Jenkins backups and code review - eedri 14:03:51 * Jenkins drive space issue. 14:03:51 * Jenkins migration from EC2 - status 14:03:51 * Push on ovirt-engine RPMs sync to ovirt.org - status 14:03:54 * Enabling Gerrit patches - everyone vs. limited 14:03:57 * All other business 14:03:59 I'd like to add puppet to the agenda 14:04:03 ok 14:04:18 i also have an update on parallel jobs 14:04:51 14[[07Infrastructure team meetings14]]4 !10 02http://wiki.ovirt.org/w/index.php?diff=3952&oldid=3948&rcid=4048 5* 03Quaid 5* (+41) 10/* 2012-07-24 */ adding new agenda items from ewoud 14:05:08 ok, first item 14:05:18 #topic Repo layout proposal for 3.1 release 14:05:33 i will start with updated status 14:05:48 thx 14:06:14 i reviewd current rpm jobs on jenkins for projects: ovirt-engine,cli,sdk,log-collector and vdsm 14:06:31 we can add more projects if it is needed 14:06:44 each job now creates both binary and src rpms 14:06:52 eedri: node rpms are created as well already 14:06:59 they're not explicit -rpm jobs 14:07:08 mburns, right, forgot about that since it wasn't in 'rpm' view tab 14:07:17 mburns, i'll add them as well 14:07:22 eedri: if you want it split, i can do that... 14:07:42 mburns, well.. let's see what is the best option and decide 14:08:02 i've taken robert suggestion for directory structure and for now each job put all rpms under: 14:08:27 rpms/3.1/nightly/fedora/17/[binary | src] 14:08:36 eedri: point of splitting is so that we can consume the src tarball for multiple distros 14:09:00 eedri: hmm, that's different than where i'm putting them 14:09:00 mburns, ok, so in that case it sounds like we need to split it 14:09:22 eedri: no plan at the moment to add any other distros for node though 14:09:52 i'm contemplating with myself... how much effort we should put in making jenkins jobs generic 14:09:59 to work on all version/ distros 14:10:09 or add new jobs per distro/ver as we progress 14:10:19 eedri: IMO, we should: 14:10:26 1. have a job that builds src rpm 14:10:29 we can always 'refactor' jobs in the future and more demand comes 14:10:41 2. separate jobs for each distro needed that builds packages based on src rpms 14:10:52 err... 14:10:58 s/src rpms/src tarballs 14:11:34 mburns, why split the job for src and binary? 14:11:59 mburns, it will duplicate much more jobs to maintain, why cant there be one job per distro to build both binary & src rpms 14:12:14 eedri: src tarball in #1, src rpm and x86_64 rpms in #2 14:12:37 mburns, ok, which projects requires tarball? other than ovirt-node 14:12:46 I agree with the split in the long term, but right now there is no distro support for non-rpm distros 14:12:51 and when we get to debian packages, have another job that takes src tarball from #1 and builds .deb packages 14:13:17 eedri: we're supposed to be posting .tar.gz or .tar.bz2 in src directory already, iiuc 14:13:37 if we're not, then we need to do that 14:13:46 mburns, for every project? 14:13:50 eedri: yes 14:13:53 mburns, along side src rpms? 14:13:56 yes 14:14:14 src rpms are really only applicable to rpm based distros 14:14:21 #action add jobs to create .tar.gz or .tar.bz2 per project in ovirt 14:14:25 and are useless for people who want to package for debian, etc... 14:14:56 ok, my idea was that each of these rpm jobs will create and deploy the rpms to a common dir 14:15:14 e.g. rpms/3.1/nightly/fedora/17 14:15:47 then the job collecting it will copy the same dir structure from all jobs 14:15:53 I think we should have {src,rpms,deb}/{3.1,3.2,stable}/ in the long run 14:16:04 where stable is a symlink to 3.2 14:16:17 eedri: that seems like we're moving a common task into multiple jobs 14:16:23 under specifics we should have whatever format the distro prefers 14:16:32 ewoud: agreed 14:16:36 mburns, i don't think so 14:16:54 mburns, the nightly task of copying and publishing the rpms to ovirt.org repos is one task 14:17:06 RobertM: not here? think you added it to the agenda 14:17:10 mburns, but each project should built its own rpms/tar independently 14:17:24 it sounds as if we have more discussion to do 14:17:24 mburns, so that people can still download rpms per commit from jenkins 14:17:25 mh, into what state do I set a bug if it is fixed by a patch from a different bug? modified? 14:17:29 eedri: correct, build is separate, but why force directory structure on the build job 14:17:29 are we going to be able to reach a consensus here? 14:17:36 Sorry was off reading up on log rotate 14:17:47 mburns, ah.. just for convience, not a hard demand 14:18:08 fabiand: yes, but send me the bz numbers as well, i want to look first 14:18:15 mburns, how would the nightly job will know where to copy the files from? 14:18:26 mburns, ok, 831668 14:18:48 mburns, let's say we'll make it generic one day and it will work on multiple jobs 14:18:57 eedri: *.src.rpm *.tar.gz *.tar.bz2 into src 14:19:23 mburns, we're planning on putting all rpms in a single flat dir? 14:19:43 then based on the %dist tag, put in fedora/17 14:19:56 mburns: I'd say .src.rpm should be under rpms, src should just be .tar.(gz|bz2|xz) 14:20:05 ewoud: sure 14:20:31 eedri: after running rpmbuild, look at the directory structure 14:20:43 mburns, it depends on the project 14:20:53 mburns, some put it under rpmtop, some under rpmbuild 14:20:56 eedri: hmm 14:21:09 eedri: ok, well, each rpm build exposes the rpms as artifacts 14:21:12 some separate src and some dont 14:21:33 so just process all the artifacts 14:21:43 the copy artifacts plugin has an option to flatten directories 14:21:51 yea, i saw that 14:22:17 so you can assume that it's a flat directory 14:22:24 ok, so agreed main night job will copy all artifacts in src & rpm dirs 14:22:58 how about http://bpaste.net/show/36543/ 14:22:59 fabiand: close nextrelease with a comment pointing to the bug/patch that fixed it 14:23:03 where src will contain *.tar(.gz|bz2) 14:23:24 Finally caught up. I think we are off track some. We are suppose to be talking about repo layouts not automating the process of getting .rpms from jenkins. Can we deal with the topic of repo layouts since we have to deal with that today. 14:23:45 RobertM: +1 on focussing on repo layout, after that how they get there 14:23:47 ewoud, exactly i want to close how the destination folder on ovirt.org will look like 14:24:09 ack 14:24:21 eedri, I think your reply suggests the 2 of use agree pretty much on the finally layout. 14:24:23 +1/-1 on my proposal here? 14:24:58 RobertM, there has been some suggestion here about diff layout for tar.gz files and src.rpm 14:25:00 ewoud, Can you please restate? 14:25:11 RobertM: http://bpaste.net/show/36543/ is the tree view 14:25:26 i think we want to invert the 3.1/3.2/etc... with the rpm/src/etc... 14:25:46 -1 14:26:15 I agree with mburns it should be 3.1/[rpm|src|deb]/ 14:27:02 I have no problem added in the extra layer of /rpm/ although I think it isn't needed 14:27:03 not sure, for gentoo it would be easy if src was just a flat directory with file names 14:27:17 * ewoud runs gentoo on his desktops 14:27:55 but you can mangle it without much difficulty 14:28:03 information architecture wise, it seems to flow better to do [release number]/[format] aka 3.1/[rpm|src|deb] 14:28:04 so I'm fine either way 14:29:08 I am fine with 3.1/[rpm|src|deb] 14:29:26 I don't like rpm/3.1/... 14:30:03 RobertM: agreed, doing rpm/3.1 says "we care more about the package format question before the version question" 14:30:04 http://bpaste.net/show/36546/ 14:30:35 quaid: I personally think most people will care more about the package format anyway 14:31:08 http://bpaste.net/show/bKqcBTNv4Svskjo8TJ3U/ 14:31:13 ewoud, Only change I would make is nightly -> 3.2 (Right now. 14:31:23 why isn't there a nightly subdir under the versions? 14:31:26 RobertM: nack on nightly-> 3.2 14:31:29 RobertM: why? nightly could be master 14:31:33 and 3.2 the branch 3.2 14:32:05 functionally nightly might be the same as a version 14:32:06 3.2 should be only populated when we have alpha/beta/rc/ga of 3.2 14:32:10 but nightly *is* a version 14:32:15 I think the top levels should correspond to git branches 14:32:16 nightly should be more like fedora rawhide 14:32:36 ok. I see the point 14:33:39 Nightly gets it own tree. 14:33:50 mburns: though 3.1 is soon to be out, and 3.2 version patches will than come in? 14:34:02 versioning will be changed 14:34:10 +1 on http://bpaste.net/show/bKqcBTNv4Svskjo8TJ3U/ 14:34:27 looking at rpms we have fedora18 around the corner as well 14:34:34 mgoldboi: yes, but those will go in nightly only until we get official builds of engine for 3.2 14:34:35 +1 on http://bpaste.net/show/bKqcBTNv4Svskjo8TJ3U/ 14:34:40 or of vdsm 14:34:57 +1 14:35:41 though not sure about el6, but that's a small detail we don't even have now 14:35:56 should it be el6.3 14:36:08 or el6/3 14:36:30 ewoud: right, we don't have deb either 14:36:34 Well el6 should be called EL or RHEL with a 6 folder under it 14:36:58 yes, EL with 6, 7, etc... 14:37:09 was just trying to show how it can be filled out later 14:37:15 but this layout should be scalable enough 14:37:31 mburns: +1 for the base layout 14:37:35 I see no -1, so can we agree? 14:37:38 And it means existing installs and files will work once new releases come out. 14:37:40 +1 14:37:42 given this, though, there is one question 14:37:49 for http://bpaste.net/show/bKqcBTNv4Svskjo8TJ3U/ 14:37:57 it only really works if there is a single srpm for el6, fc17, fc16, etc... 14:38:14 right 14:38:21 srpm should have the same subdirs as rpm 14:38:26 mburns: you're right, I placed srpm under the distro on the same level as noarch/x86_64 14:38:36 just w/o the arch 14:38:59 ok, so move srpm to same level as noarch/x86_64/etc 14:39:09 +1 14:40:18 Under the /fedora/17/srpm ? 14:40:25 RobertM, yes 14:40:37 http://bpaste.net/show/QwiYrrY5jvXP22A12SBM/ 14:40:42 that should be final 14:41:24 I have no problem with that. .srpm files seem to be version spefic. 14:41:28 one more thing 14:41:38 how do we keep history? 14:41:52 does this layout should keep one version or many? 14:42:04 eedri: always keep GA'd versions 14:42:25 eedri: and *always* keep .tar.gz/.tar.bz2 14:42:26 mburns, so each dir will contain X versions? 14:42:34 yes 14:42:48 That would be fine. 14:42:52 eedri: otherwise, no need to keep RC/beta/nightlies 14:42:58 mburns, ok ,we'll need to set up a cleaner with retention policy 14:43:05 this might take up some space, but if we get to that I think we can find some mirrors for it 14:43:54 I know the admins of ftp.nl.debian.org who also host a mozilla mirror who will gladly use up some more bandwidth 14:45:09 I think their current record is close to 8Gb/s and they'll get pie from the university if they reach it 14:45:15 should there be a jenkins job to deploy stable (non nightly) rpms ? will it be done manually? 14:45:16 Part of the reason I wanted a few days worth of nightly's Goto love mirrors :) 14:45:41 eedri: no, that should be done manually 14:45:43 mburns, thanks 14:46:02 mburns, I think they should be done in Jenkins but manually 14:46:09 RobertM, +1 on that 14:46:35 RobertM: so a locked down job that only a few people can use? 14:46:43 Then it doen't becomes mburns full time job to do new beta releases 14:47:01 mburns, Yes 14:47:01 * quaid looks at /topic & agenda & time left 14:47:15 this has been a good discussion, but I wonder if we should wrap it up? Or ...? 14:47:19 I think we have 4 of 6 topics going on all at once. 14:47:24 quaid: +1 on wrapping 14:47:37 RobertM: so essentially make koji style jobs that build the official rpms for release 14:47:38 Lets make the call on repo layout put that in the log then we can deal with Jenkins 14:47:45 mburns, Yes 14:47:46 ack 14:48:03 I think we've discussed the repo layout and reached the conclusion 14:48:16 vote on http://bpaste.net/show/QwiYrrY5jvXP22A12SBM/ (pasted by mburns before)? 14:48:21 +1 14:48:33 Were is the action item? 14:48:40 * mburns wonders if it's bad form to nack his own proposal... 14:48:41 +1 14:48:42 Where is the action item. 14:48:54 the action item should be to create this stucture on ovirt.org 14:49:05 Also we need to move to the next item then 14:49:19 and to run a script (locally/jenkins) to create a repo for the files in it 14:49:32 at least for the nightly dir 14:49:35 I see no, -1, so we all agree on this layout? 14:49:41 +1 for layout 14:49:46 if so we get #agree and #action from it ;) 14:50:06 Ok add an action item for me to put the new repo layout into place 14:50:25 +1 for rmiddle 14:50:37 and we can move on. 14:50:38 #char ewoud eedri RobertM mburns 14:50:43 #chair ewoud eedri RobertM mburns 14:50:43 Current chairs: RobertM eedri ewoud mburns quaid 14:50:47 (so you can do #agreed) 14:51:00 14[[07Repo layout14]]4 !N10 02http://wiki.ovirt.org/w/index.php?oldid=3953&rcid=4049 5* 03Ekohl 5* (+684) 10Created page with "
 . ├── 3.1 │   ├── deb │   ├── rpm │   │   ├── EL │   │   │   ├── 6 │   │   │   └── 7 │   │   └── fedora ..."
14:51:03  #action RobertM to put new repo in place
14:51:10  #agree layout as on http://wiki.ovirt.org/wiki/Repo_layout
14:51:20  ewoud: +d
14:51:39  #agreed layout as on http://wiki.ovirt.org/wiki/Repo_layout
14:51:49  did we reach any other decisions in that discussion?
14:51:57  #agree  layout as on http://wiki.ovirt.org/wiki/Repo_layout
14:51:58  or proposed decisions?
14:52:13  quaid, there are still issues left for disscusion
14:52:31  quaid, not sure we'll be able to cover them all
14:52:36  I think we should take how we get the files into the repo to the mailing list
14:52:39  given the time
14:52:48  lets take jenkins stuff to the list
14:52:52  Next on agenda is backups of Jenkins?  Do you guys just want me to rsnapshot the /var/lib/jenkins folder nightly?
14:53:03 * eedri suggests using scp or artifact deployer jenkins plugins
14:53:11  eedri: mind writing a proposal on the mailinglist?
14:53:12  #topic Jenkins backups
14:53:20  ewoud, sure
14:53:36  #action eedri to write a proposal on the list about how to get the files in to the repo
14:54:11  RobertM, this folder contains all the buils, it can be a few dosens GB
14:54:21  RobertM, rsnapshot knows how to calc diffs?
14:54:22  eedri, I have the space.
14:54:32  RobertM, ok great
14:54:40  eedri, Yes.  It is based on rsync
14:54:53 * eedri suggests moving all code in jobs to jenkins git repo
14:55:32  this will serve also as backup and make management and code review more simple
14:56:08  +1, afaik they're plain text files so git should be well suited
14:56:16  eedri, Do you know how to do that?  I don't see official support for that but I could be missing a plug-in
14:56:20  ewoud, 99% of it is bash
14:56:24  Hi all
14:56:28  RobertM, already doing it
14:56:38  RobertM, only way to scale in jenkins
14:56:39  Sorry to disrupt... anyone know where I might get a hold of oschreib?
14:56:49  dneary, he left
14:56:55  hmm...
14:57:03  there are plugins for svn (i haven't seen git)
14:57:09  eedri, For the day, or left Red Hat?
14:57:10  eedri: is there an auto-deploy?
14:57:11  mburns, scm git
14:57:18  mburns, multiple scm plugin
14:57:34  mburns, what do you mean ? for backup?
14:57:51  eedri: maybe i'm misunderstanding
14:57:52  mburns, no, i mean we should treat jenkins code as we treat any other ovirt project
14:57:55  where are changes made?
14:58:02  in git first? or through jenkins?
14:58:05  mburns, move all code into a git repo
14:58:27  mburns, when you want to run it, you add the git repo to the job (via multiple scm plugin), alongside ovirt repo
14:58:36  mburns, clone it to a subdir (i.e. jenkins)
14:58:43  mburns, and use the code from the repo
14:58:55  Need to research multiple SCM plugin before he could vote.
14:59:06 * mburns not sure he likes this, actually
14:59:16  i agree with backing up config
14:59:26  mburns, let's say you have 10 similar jobs
14:59:41  mburns, where you need exactly the same code , maybe with a slight change
14:59:50 * mgoldboi got to go
14:59:56  Lets table moving Jenkins management to git and move it to the list
14:59:59  but not sure about having to commit stuff directly into git, get it approved and pushed, before it can be activated in jenkins
15:00:01  mburns, isn't it more logical to use code from a git repo instead of just wirting it in the job?
15:00:20  mburns, i would vote that each one of infra will have +2 for jenkins repo
15:00:34  eedri: yes, and i've done that in the past, but the script is kept in ovirt-node-iso git repo, not in a generic jenkins repo
15:00:36  It sounds like we aren't clear on all the aspects yet so it is to soon to vote.
15:00:37  mburns, and only if that change is critical or affects all jobs it will need a rebiew
15:01:00  Can I get a vote on backing up Jenkins to my location using rsync/rsnapshot?
15:01:02  +1 on tabling this for now
15:01:05  eedri: would it be a repo per jenkins job or 1 global config?
15:01:06  mburns, i dont have a problem with seperate jenkins folder/repos per project
15:01:07  RobertM: ack on backing up
15:01:21  RobertM: should be the good short term solution
15:01:22  ewoud, not pet job, maybe per ovirt project
15:01:32  eedri: i meant that i have a jenkins.sh in the base ovirt-node repo
15:01:40  and the build in jenkins just calls that script
15:01:44  mburns, i don't like that method
15:01:53  mburns, jenkins code shouldn't be part of the project repo
15:01:58  mburns, it's infra
15:02:18  eedri: i disagree to a certain extent, but let's table this for now
15:02:26  mburns, ok
15:02:28  quaid, can you add an action item on me backing up the system using rsync for now and moving talk about managing jenkins using git to list.
15:02:36  anyone can make an action
15:02:36  +1 on backing up
15:02:58  #action RobertM backup /var/lib/jenkins using rsync
15:03:16  RobertM, you should exclude /var/lib/jenkins/workspaces from that
15:03:16  next item or do people need to leave?
15:03:46 * mburns has another call, but don't block on me
15:03:49  eedri, Noted
15:03:53  RobertM, might be good to backup /var/log/jenkins as well
15:04:48  danken: dougsland:  fyi:  https://bugzilla.redhat.com/show_bug.cgi?id=842775
15:05:38  #topic enabling per patch builds on Jenkins
15:06:42  14[[07Infrastructure team meetings14]]4 !10 02http://wiki.ovirt.org/w/index.php?diff=3954&oldid=3952&rcid=4050 5* 03Ekohl 5* (+0) 10Eedri added parallel jobs to the agenda
15:06:44  this topic is blocked by the security issues we raised on list, did we agreed on the process of approving ?
15:06:50  eedri, When you said "he left", did you mean for the day?
15:07:14  dneary, i think he's in the gym.. might be back in an hour
15:08:07  eedri: I'm not sure what was agreed on the list, looking up now
15:10:07  Unless anyone has anything important I suggest we table to rest of the agenda and call the meeting over?
15:10:32  I'll prepare something for puppet on the mailing list
15:10:51  RobertM, agree
15:10:54  #action ewoud prepare a puppet proposal on the mailing list for the next agenda
15:11:10 * quaid is running a phone meeting now, attention very split
15:11:49  quaid, Lets just end this sucker so am I.
15:12:19  I would like to propose that everyone who wants to add something to the agenda prepares the item a bit for a meeting
15:12:34  I'm fine with discussion a proposal on IRC, but meetings should be kept short imho
15:12:44  agreed
15:12:56  +1
15:13:27  fwiw, anyone with chair can close, etc.
15:13:34  closing in 30 seconds
15:13:37  #agreed new agenda items should be prepared by the proposer so we reach a conclusion faster
15:13:47  I'm done now :)
15:16:24  eedri, Ah, right
15:16:43  eedri, When you said "he left", I thought you might have meant "he left Red Hat"
15:17:17  dneary, :) too dramatic
15:17:59  since quaid didn't do it yet
15:18:00  #close
15:18:13  #endmeeting