14:04:53 #startmeeting weekly infra 14:04:53 Meeting started Mon May 20 14:04:53 2013 UTC. The chair is ewoud. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:04:53 Useful Commands: #action #agreed #help #info #idea #link #topic. 14:05:01 Agenda from http://www.ovirt.org/Infrastructure_team_meetings#2013-05-13 14:05:06 * eedri has to leave soon, will catch up on the summary 14:05:08 Introductions 14:05:08 Review of action items 14:05:08 Hosting 14:05:08 rackspace01 installed? 14:05:08 Puppet 14:05:11 Jenkins 14:05:13 Other business? 14:05:16 Trac review 14:05:20 #chair knesenko eedri 14:05:20 Current chairs: eedri ewoud knesenko 14:05:39 ewoud: I'm here too :) 14:05:47 #chair dcaro 14:05:47 Current chairs: dcaro eedri ewoud knesenko 14:06:13 #topic Introductions 14:06:56 eedri: you mentioned a potential new member in past meetings; anything to share already? 14:07:24 ewoud, i was reffering to knesenko 14:07:46 eedri: check 14:07:54 then I thinkw e have no further introductions 14:08:03 eedri: maybe didi wants to join ? 14:08:44 if he's interested he's welcome to join the meetings and start getting involved 14:09:06 * obasan here 14:09:33 #chair obasan 14:09:33 Current chairs: dcaro eedri ewoud knesenko obasan 14:09:44 didi: hey, are you interested to join the infra meeting ? 14:10:20 ewoud: lets continue ... I"ll talk with him after the meeting 14:10:31 knesenko: ok 14:10:36 #topic Review of action items 14:11:20 I just realise I forgot to update the agenda on the wiki 14:11:27 knesenko, there is already a meeting 14:11:29 I am here too, hi 14:11:38 knesenko, disregard my last comment 14:11:56 eedri: I am confused :) 14:12:27 knesenko, nm 14:12:32 ewoud, please continue 14:12:49 eedri: I was looking for the action items 14:13:03 Rydekull Add the jenkins slaves on a list on the wiki quaid look for compatible times for a docs sprint obasan look into icinga for monitoring dcaro ensure VM can be created in foreman using ovirt on alterway02 ewoud write a design proposal for the puppet repository 14:13:07 derp 14:13:31 * quaid look for compatible times for a docs sprint 14:13:31 * obasan look into icinga for monitoring 14:13:31 * dcaro ensure VM can be created in foreman using ovirt on alterway02 14:13:31 * ewoud write a design proposal for the puppet repository 14:13:39 didi: hey Didi ... you are in the middle of the ovirt infra meeting . This small team maintains upstream infra . please join the meeting if it sounds interesting for you 14:14:16 ewoud, i have to leave, dcaro can update on rackspace status 14:14:24 didi: just observing is fine as well 14:14:25 eedri: ok 14:14:34 ok 14:14:41 ewoud, just a small update on jenkins 14:14:48 Is the meeting in IRC only? 14:14:54 dcaro: did you check foreman + ovirt on alterway02? 14:14:56 didi: yes 14:14:57 didi: yes 14:15:06 ewoud, add 2 new jobs - one for open tasks that send monthly report and one to scan warnings on ovirt-engine 14:15:14 I have read some documentation about icinga. as far as I understood. it's 100% compatible with nagios's configuration files. which will make it easier to maintain. it's basically contains a nicer gui and some other features. if I'm given a server I will install it for us. I have prior experience with nagios 14:15:32 ewoud: no, sorry, I was frustrated trying to install the rackspace servers 14:15:34 obasan: cool 14:15:42 dcaro: ok 14:15:49 * eedri leaving, catch up later 14:15:54 eedri, bye 14:15:58 last week I was at djangoconf so I had no time either 14:16:37 let's continue with hosting then 14:16:40 #topic hosting 14:16:40 hey guys , please take a look on this - https://apps.fedoraproject.org/ 14:16:54 would you like to have it for oVirt infra as well ? 14:17:14 knesenko: that looks much cooler than a wiki page 14:17:47 ewoud: I already created one for internal usage .... quite simple to deploy 14:17:59 ewoud: so I can create one for oVirt infra as well 14:18:50 knesenko: it looks like static files that are generated, so hosting should be simple? 14:19:27 ewoud: yes correct . its a static yaml that generates html 14:20:05 ewoud: its very simple and will be nice to have it for oVirt 14:20:09 +1 from me 14:20:37 +1 14:20:51 +1 14:20:58 #action knesenko set up something similar to https://apps.fedoraproject.org/ for ovirt 14:21:02 ok so no objections 14:21:17 +1 14:21:24 it's really cool 14:22:08 dcaro: any news on rackspace? 14:23:03 submitted a ticket for it - https://fedorahosted.org/ovirt/ticket/49#comment:1 14:23:29 ewoud: I was unable to install, I've tried using fedora, ubuntu, centos but all have the same issues 14:23:53 ewoud: I have an open ticket with them, and have tried changing some options but no luck yet 14:24:03 ewoud: I'll keep insisting 14:24:41 dcaro: what makes it so hard? still the VPN? 14:24:42 What is the problem with rackspace? 14:25:15 ewoud: nono, the iDrac freezes on all the modes except safe graphics, but in safe graphics mode the keyboard and the mouse stop working :S 14:25:47 didi: we have 2 phyiscal machines at rackspace we intend to use as jenkins slaves, but we're having trouble installing them 14:26:12 what about a text install? 14:26:13 dcaro: and text installer? 14:26:14 damn these windows updateservers are slow 14:26:20 lots of disconnects 14:26:23 didi: ewoud: freezes too... 14:26:25 ewoud: he man! 14:26:33 perhaps serial console? 14:26:41 YamaKasY1: hello 14:26:45 dcaro: tried a kickstart? 14:26:57 ewoud: free today I hope ? 14:27:06 ewoud: no, not yet 14:27:19 YamaKasY1: yep 14:27:35 ewoud: nais 14:28:33 dcaro: if you need help with a kickstart, please let us know 14:29:10 question regarding - https://fedorahosted.org/ovirt/ticket/47 14:29:16 can someone help me out with adding a NAS ISO domain? 14:29:17 ewoud: we need some good docs on foreman and so on... deployment is key for ovirt 14:29:24 or just point me to a doc perhaps? 14:29:35 I would like to push it forward , but I am stuck . 14:29:49 im having touble adding the iso domain that is located on a NAS and not the ovirt-engine 14:29:51 where do we want to backup it ? and how ? rsync, nfs ? 14:30:06 or should it be mounted on the ovirt-engine? 14:30:10 YamaKasY1: which side of the integration are you looking at? because installing VMs on ovirt using foreman is easy; not sure about installing ovirt nodes using foreman 14:30:23 jmosco: you need to add it from the webadmin 14:30:28 knesenko: good question 14:30:48 ok, I will try 14:30:52 ewoud: VM's 14:31:00 ewoud: are there any good docs than ? 14:31:06 ewoud: patch is ready - http://gerrit.ovirt.org/#/c/14508/ . 14:31:13 * YamaKasY1 hates puppet to setup actually... 14:31:14 knesenko: I'd say a server on a different phyical location (so not alterway02), but linode01 is already rather full 14:31:16 so Datacenters, Default, storage, add domain? 14:31:24 but need to decide to which server do we want to backup 14:31:33 YamaKasY1: poke me after the meeting, using the installer it's not that hard 14:31:38 jmosco: yes, choose ISO domain there 14:31:44 ewoud: ok will do 14:31:53 ewoud: when are you here ? 14:32:02 5am ? 14:32:05 pm 14:32:06 I mean 14:32:08 infra meeting usually ends around 5pm yes 14:32:26 knesenko: how big are the backups? 14:32:38 ewoud: hmmm .... have no idea . 14:32:46 ewoud: need to check this out ... 14:33:01 ewoud: please continue ... i"ll check 14:33:17 ewoud: ok thanks 14:33:26 knesenko: ok 14:33:38 so I'd like to start installing VMs on alterway02 14:34:06 we originally had the plan to start migrating services from linode, but we also have a need for monitoring 14:34:31 and we do not how know how stable alterway02 currently is so I'd propose to start setting up a new service rather than migrate 14:34:42 that would also mean obasan could start setting up icinga 14:35:19 sounds good 14:35:35 makes sense 14:35:49 do we need a ticket on icinga ? 14:35:53 knesenko: ok, so I would choose import domain instead? 14:35:55 installation 14:36:06 knesenko: could be good 14:36:19 jmosco: if it is an existing domain that you are using with another ovirt instance - so yes 14:36:33 dcaro: shall I take over setting up foreman further or do you want finish it? 14:36:37 import - is to import an existing ISO domain . new - is to create a new ISO domai n 14:37:04 * knesenko opening a ticket to obasan 14:37:22 ok. The ISO domain was created on installation of the engine, but there isnt anything there because it was not local, its on a NAS 14:37:24 ewoud: I'd like to give it a push 14:38:02 dcaro: ok 14:38:27 obasan: assign it to yourself - https://fedorahosted.org/ovirt/ticket/50 14:38:34 ewoud: ^^ the ticket 14:38:41 knesenko: I think you also noticed that the LV which houses the VMs on alterway02 was rather small when we installed foreman, will that become a problem? 14:39:01 ewoud: yes it is ! 14:39:14 ewoud: we need more space for the VMs there . 14:39:36 ewoud: I am not sure we can extend it 14:39:39 :\ 14:39:46 knesenko: we have space in the VG, but does ovirt like that? 14:40:20 knesenko: foreman can be turned off so maybe we can just unlink it, resize and re-attach? 14:40:21 ewoud: I"ll need to check with the storage guys 14:40:52 knesenko: please do 14:41:05 #action knesenko check with the storage guys how we can resize the LV on alterway02 14:41:44 ewoud, I need full access to alterway2 in order to install an icinga instance on it 14:42:09 obasan: I was hoping that we could use the foreman-ovirt integration to do so 14:42:34 ewoud: what do you mean by 'foreman-ovirt integration' ? 14:42:44 obasan: I already set it up far enough that it will create a VM for you, give you a console 14:42:54 obasan: that can be the 'excuse' to get me going on setting up foreman :) 14:43:08 knesenko: foreman can provision a VM in ovirt for you 14:43:10 im a little confused with this. There is a ISO domain in the top level "system", but not in the default data center 14:43:15 should that be removed? 14:43:15 ewoud, if I have a vm with full root access > I'm good with it :) 14:43:31 ewoud: aha ok 14:43:50 that reminds me, we may need an LDAP server or something similar at some point I think 14:44:06 I'd rather not share a single admin account 14:44:09 jmosco: sorry , but I can't understand the case 14:44:22 in foreman we could use local accounts, but it would be nice if we could also have personal accounts in ovirt 14:44:29 ok, I am trying to explain, im very new to this 14:44:39 ewoud: do we have enough resources for all these services that we want to install ? 14:44:41 :) :) 14:44:51 jmosco: go for it ! :) 14:45:04 the ISO share im trying to add 14:45:08 #action dcaro ensure we can provision a VM on ovirt using foreman 14:45:23 ewoud: dcaro what about puppet ? 14:45:35 there is the default datacenter, but the share seems to be on the top level, "System" under storage 14:45:42 I cant add it to the default datacenter 14:45:50 knesenko: in what context? 14:46:09 I will also need an account to alterway02's userportal or webadmin to access the machine. 14:46:14 then I go to the "Default" datacenter, then storage, there is nothing there 14:46:17 knesenko: ewoud: I forgot, ewoud, did you send the code that you wanted to use for the users and stuff? 14:46:23 ewoud: we need a puppet class that will create users with the same uids and copy ssh keys ... etc 14:46:25 I cant add the ISO because it complains that its already there 14:46:27 dcaro: I did not :S 14:46:43 ewoud: ok, not just me then :) 14:46:51 knesenko: yes, that should be done 14:47:03 jmosco: do have hosts in that DC and storage domain ? 14:47:09 ewoud: ok 14:47:16 #action ewoud draw up an initial puppet design proposal 14:47:18 host yes 14:47:30 guys should the network statics for nics work and show traffic ? 14:47:34 storage, no 14:47:44 when I try to import it, I get this: Import operation failed. Domain ISO-SHARE already exists in the system.Import operation failed. Domain ISO-SHARE already exists in the system. 14:48:00 knesenko: and I don't think an LDAP server requires very much, but it gets interesting in where to host 14:48:30 ewoud: I have no experience with LPAD installations 14:48:32 I'd say that it should live on the physical host because you can't rely on VMs 14:48:59 didi: I think that you have some experience with LDAP servers , right ? 14:49:04 because if you need to access it and you can't log in to ovirt because the LDAP server is down ... 14:49:07 knesenko: not really 14:49:33 but I think we can work around that for a bit if we rely on foreman to provision VMs for now 14:49:34 ewoud: but you have the local user 'admin' 14:49:55 knesenko: true, but eventually I'd like to move away from that as much as possible 14:50:15 ewoud: we can use in case LDAP service will be down :) 14:50:36 the default datacenter also says its uninitialized 14:50:46 knesenko: yes, that's why I think we'll eventually need LDAP 14:51:16 jmosco: so you can't add ISO domain . fist you need to add storage domain to the DC 14:51:23 anyway, I'd like to talk a bit about puppet 14:51:34 ewoud: ok . 14:51:40 #topic puppet 14:52:01 right, so this is a bit of what I want to write in the introduction mail 14:52:21 but we need to make some choices on where to store the git repo 14:53:03 currently on puppet@foreman.ovirt.org:puppet.git there's an empty repo with a post-receive hook that updates /etc/puppet/environments which translates each branch into a checkout 14:53:09 ugh, this is confusing 14:53:40 what is the System folder storage ISO-SHARE doing then? 14:53:41 in the short run that'll be fine, but I think we will want to move that to gerrit at some point 14:53:44 and should that be removed? 14:54:34 ewoud: I think we should put it unger our gerrit instance, but that will force us to use gerrit hooks instead of the ones already in place (not a big deal though) 14:55:15 jmosco: I have no idea. 14:55:21 dcaro: and where does the output of those hooks go? 14:55:21 jmosco: is that a clean install ? 14:55:25 yes 14:55:38 jmosco: allinone ? 14:55:54 no 14:56:01 engine on a different host 14:56:02 dcaro: the post-receieve hook has no concept of status codes to see if everything went OK nor any transaction mgmt or something 14:56:06 ewoud: by default nowhere... but with the dispatcher to gerrit.hooks.log 14:56:21 jmosco: did you add new host/s to the setup ? 14:56:22 the host I imported is in the default datacenter 14:56:28 yes 14:56:30 dcaro: ok 14:57:17 jmosco: can you see that host in the webadmin ? its up and running ? 14:57:20 do you guys think we should already install puppet on all hosts even though there aer no classes yet? 14:57:23 yes 14:57:39 ewoud, +1. 14:57:46 ewoud, they will eventually be managed with puppet 14:57:47 ewoud: yes, at least that way we have them in foreman 14:57:58 jmosco: did you add new data storage domain to the DC ? 14:58:11 ewoud: dcaro +1 14:58:12 I tried, but it failed 14:58:21 ok, then I'll set that up 14:58:24 it seems to be in the "System" menu 14:58:27 #action ewoud add all hosts to puppet 14:58:47 the storage option, its there but has a red slash 14:58:48 jmosco: do to the storage tab .... clikc new data domain . 14:59:04 dcaro: you have experience with exported resources, right? 14:59:17 knesenko: yes, but I cant add iso domain there, just data / nfs 14:59:22 I'd like to use puppetdb, but I'm afraid it will use quite a bit of resources 14:59:27 ewoud: not extensive but yes, I've played with it 14:59:36 I guess I can delete the one thats in the "System" menu? 14:59:53 jmosco: first you need to add data domain . having ISO domain - make no sense without data 15:00:00 ewoud: I don't think that we have enough hosts to have performance problems with that 15:00:00 at $employer we use it to fill munin and looking into using it to fill nagios as well 15:00:16 direct from puppetdb? 15:00:20 jmosco: after adding a data domain - you should see the ISO domain option 15:00:30 ok, ill try that 15:00:31 dcaro: export to puppetdb 15:00:33 thank you 15:00:40 sorry about the newbie stuff 15:00:42 ewoud: okok 15:01:28 though I did hear that at ebay they have a script that just reads puppetdb directly and fills their nagios server config since it performs better 15:01:38 jmosco: please read this one - http://www.ovirt.org/Quick_Start_Guide 15:01:59 btw, I will also send out an email about the meeting time 15:02:16 weird, I have so slow I/O over 1GB on a raid10 sata set, only on windows it seems 15:02:18 Rydekull has other meetings, eedri as well and it's not great for me either 15:02:28 ewoud: yep, I've read about it to 15:02:30 the freenas filer has a lagg of 3GB 15:02:55 anyway, I'd like to end the meeting now 15:03:35 ewoud: ok 15:03:49 unless you have items we need to discuss, but we ran out of time 15:04:13 ewoud: yes we are . we will discuss them on mailing list then 15:04:39 going once 15:04:45 going twice 15:04:55 #endmeeting