Historical diskspace to formatted Excel spreadsheet

Author Message
ginolard

  • Total Posts : 1347
  • Scores: 23
  • Reward points : 0
  • Joined: 8/11/2005
  • Status: offline
Historical diskspace to formatted Excel spreadsheet Tuesday, June 01, 2010 8:06 PM (permalink)
0
Oohhh boy.  This is going to be a toughie.

Now that I am, apparently, the resident "Powershell expert" in my team (i.e. I know GET-HELP), I'm being tasked with more and more complicated stuff.  This latest one is going to be a real challenge.

I have to get the free diskspace on every local partition of every server.  That's easy enough with WMI.  However, this then has be put into an Excel spreadsheet which keep 3 months worth of data (the script will be once a month).

So, I'm not sure the best approach here.  Put the data in a CSV first (which will be updated every month) and then use Excel's COM object to input and format it?

How to maintain the historical data?
Author of ManagePC - http://managepc.net

 
#1
    ginolard

    • Total Posts : 1347
    • Scores: 23
    • Reward points : 0
    • Joined: 8/11/2005
    • Status: offline
    Re:Historical diskspace to formatted Excel spreadsheet Tuesday, June 01, 2010 9:14 PM (permalink)
    0
    The more I think about this, the harder it gets.  If the list of servers never changed, that would be fine. I could just delete the oldest data for each server add the newest data.

    However, we are constantly adding new servers at the moment so I will have to query AD for all Windows servers each time and then fetch the free diskspace data for each one.  How then to delete the old data for each existing server? 

    Argh.  It's making my head hurt.

    I'm thinking this is database material and not Excel.....
    Author of ManagePC - http://managepc.net

     
    #2
      ebgreen

      • Total Posts : 8227
      • Scores: 98
      • Reward points : 0
      • Joined: 7/12/2005
      • Status: offline
      Re:Historical diskspace to formatted Excel spreadsheet Wednesday, June 02, 2010 2:55 AM (permalink)
      0
      I have been wrestling with similar issues and Excel. I actually do all of the things that you mention plus some. I have to produce daily spreadsheets that show month to date tickets for several support groups. Here is how I do it:

      An excel report is produced each day with the ticket information from the ticketing system
      My script uses the COM object to convert the .xls file to a .csv file
      Import-CSV gets all the data into memory where I can play with it.
      I go through the data categorizing it and doing all the calculations that I need.
      I use the COM object to open a template spreadsheet that has a summary tab with all the pretty formatting on it.
      I use Export-CSV to spit out a csv file with the data for each of the groups.
      I use the COM object to add a tab to the template for each of the groups.
      I use the com object to import the associated CSV for each of those tabs.
      I use the COM object to go through the cells on the summary tab adjusting the formulae to point to the appropriate group tabs.
      "... when you are good and crazy, oooh, oooh, oooh, the sky is the limit!" - The Tick
      Goog places to start:http://www.visualbasicscript.com/m_24727/tm.htm
      http://www.visualbasicscript.com/m_47117/tm.htm
       
      #3
        ginolard

        • Total Posts : 1347
        • Scores: 23
        • Reward points : 0
        • Joined: 8/11/2005
        • Status: offline
        Re:Historical diskspace to formatted Excel spreadsheet Wednesday, June 02, 2010 3:04 AM (permalink)
        0
        Wrestling is right.  I spent a good couple of hours on it before speaking to our DBA and asking him how easy it would be use a few tables in an Oracle server somewhere.  I think that's going to be our best bet.  It's just so much easier to do it that way.
        Author of ManagePC - http://managepc.net

         
        #4
          ginolard

          • Total Posts : 1347
          • Scores: 23
          • Reward points : 0
          • Joined: 8/11/2005
          • Status: offline
          Re:Historical diskspace to formatted Excel spreadsheet Thursday, June 03, 2010 2:53 AM (permalink)
          0
          How come I can post a single line response like this but if I try and post some PS code and an explanation of my solution it just sits there on "waiting for reply..."


          <message edited by ginolard on Thursday, June 03, 2010 2:55 AM>
          Author of ManagePC - http://managepc.net

           
          #5
            ginolard

            • Total Posts : 1347
            • Scores: 23
            • Reward points : 0
            • Joined: 8/11/2005
            • Status: offline
            Re:Historical diskspace to formatted Excel spreadsheet Thursday, June 03, 2010 5:54 PM (permalink)
            0
            Righto.  I've got it.  I've created a db in SQL Server containing 3 tables (servers, partitions and diskspace).  I then get the diskspace info and write it to a CSV file in the following format

            SERVERNAME,PARTITION,SIZE,USED
            SERVER1,C:,100,50
            SERVER1,D:,500,350
            SERVER2,C:,100,25
            SERVER2,D:,1000,500
            SERVER2,E:,1000,100

            And then, the wonderful IMPORT-CSV cmdlet lets me read in the data.  God, I love Powershell!

                 
             #Open a connection to the database 
             $conn = New-Object System.Data.SqlClient.SqlConnection("Data Source=127.0.0.1\SQLEXPRESS; Initial Catalog=DISKSPACE; Integrated Security=SSPI")     
             $conn.Open()     
             $cmd = $conn.CreateCommand()     
             
             #Read the diskspace data 
             $data=import-csv c:\temp\diskspace.csv      
             
             #Iterate the data and define variables for the required fields 
             foreach ($line in $data) {     
                 $server=$line.server     
                 $partition=$line.partition     
                 $size=$line.Size     
                 $used=$line.Used     
                 $date=get-date -format "yyyy-MM-dd"      
                      
                 #Update the servers table with any new servers (i.e. ones that don't exist already) 
                 $cmd.CommandText ="INSERT Servers (Servername) SELECT '$server' WHERE NOT EXISTS (SELECT id FROM Servers WHERE Servername = '$server')"     
                 $cmd.ExecuteNonQuery()     
                      
                 #Update the partitions table with the partitions 
                 $cmd.CommandText="Insert Partitions (ServerID,Partition) Select id, '$partition' from Servers Where ServerName='$server'"     
                 $cmd.ExecuteNonQuery()     
                      
                 #Add the diskspace data 
                 $cmd.CommandText="Insert Diskspace (PartitionID,Size,Used,ImportDate) Select id, '$size','$used', '$date' from Partitions Where (Partition = '$partition' AND ServerID=(Select id from Servers Where ServerName='$server'))"     
                 $result=$cmd.ExecuteNonQuery()     
                      
             }     
             


            I can use calculated columns in DB to work out the free space and any percentages.

            The tables are defined as follows

            Server table

            Column1 = id (Primary Key) - integer value set to IDENTITY
            Column2 = ServerName (nvarchar)  Unique constraint set on this column (we don't need the same server added more than once!)

            Partitions Table

            Column1 = id (Primary Key) - integer value set to IDENTITY
            Column2 = ServerID (int) - related to Server.id
            Column3 = Partition (nvarchar)

            UNIQUE constraint on Column2 + Column3 so that we don't add the same partition for each server more than once

            Diskspace table

            Column1 = id (Primary Key) - integer value set to IDENTITY
            Column2 = PartitionId (int) = related to Partitions.id
            Column2 = Size (decimal 18,2)
            Column3 = Used (decimal 18,2)
            Column4 = ImportDate (date)
            Column5 = Free (computed to Size-Used)
            Column6 = %Free (computed to ((Size-Used)/Size)*100
            Column7 = %Used (computed to (Used/Size) * 100

            Strictly speaking, I don't need to use that complicated INSERT statement for adding the servers (i.e. with the WHERE EXISTS clause) because of the UNIQUE constraint on the ServerName column.  I could just attempt to add all servers found and let the DB reject any duplicate records but I prefer the "better safe than sorry" approach ;)

            <message edited by ginolard on Thursday, June 03, 2010 6:12 PM>
            Author of ManagePC - http://managepc.net

             
            #6
              ebgreen

              • Total Posts : 8227
              • Scores: 98
              • Reward points : 0
              • Joined: 7/12/2005
              • Status: offline
              Re:Historical diskspace to formatted Excel spreadsheet Thursday, June 03, 2010 7:35 PM (permalink)
              0
              Nice! Wish using a DB was an option for me.
              "... when you are good and crazy, oooh, oooh, oooh, the sky is the limit!" - The Tick
              Goog places to start:http://www.visualbasicscript.com/m_24727/tm.htm
              http://www.visualbasicscript.com/m_47117/tm.htm
               
              #7
                ginolard

                • Total Posts : 1347
                • Scores: 23
                • Reward points : 0
                • Joined: 8/11/2005
                • Status: offline
                Re:Historical diskspace to formatted Excel spreadsheet Thursday, June 03, 2010 10:52 PM (permalink)
                0
                Access?
                Author of ManagePC - http://managepc.net

                 
                #8
                  ebgreen

                  • Total Posts : 8227
                  • Scores: 98
                  • Reward points : 0
                  • Joined: 7/12/2005
                  • Status: offline
                  Re:Historical diskspace to formatted Excel spreadsheet Friday, June 04, 2010 12:50 AM (permalink)
                  0
                  I would almost rather use Excel than sink into the shady underworld of the Access DB. . I could go that route, but this grew organically (like all the worst code does) and it is pretty much complete using Excel.
                  "... when you are good and crazy, oooh, oooh, oooh, the sky is the limit!" - The Tick
                  Goog places to start:http://www.visualbasicscript.com/m_24727/tm.htm
                  http://www.visualbasicscript.com/m_47117/tm.htm
                   
                  #9
                    ginolard

                    • Total Posts : 1347
                    • Scores: 23
                    • Reward points : 0
                    • Joined: 8/11/2005
                    • Status: offline
                    Re:Historical diskspace to formatted Excel spreadsheet Friday, June 04, 2010 2:11 AM (permalink)
                    0
                    Somehow I just knew you'd be averse to using Access ;)
                    Author of ManagePC - http://managepc.net

                     
                    #10
                      ebgreen

                      • Total Posts : 8227
                      • Scores: 98
                      • Reward points : 0
                      • Joined: 7/12/2005
                      • Status: offline
                      Re:Historical diskspace to formatted Excel spreadsheet Monday, June 07, 2010 12:09 AM (permalink)
                      0
                      I try to be consistent.
                      "... when you are good and crazy, oooh, oooh, oooh, the sky is the limit!" - The Tick
                      Goog places to start:http://www.visualbasicscript.com/m_24727/tm.htm
                      http://www.visualbasicscript.com/m_47117/tm.htm
                       
                      #11

                        Online Bookmarks Sharing: Share/Bookmark

                        Jump to:

                        Current active users

                        There are 0 members and 1 guests.

                        Icon Legend and Permission

                        • New Messages
                        • No New Messages
                        • Hot Topic w/ New Messages
                        • Hot Topic w/o New Messages
                        • Locked w/ New Messages
                        • Locked w/o New Messages
                        • Read Message
                        • Post New Thread
                        • Reply to message
                        • Post New Poll
                        • Submit Vote
                        • Post reward post
                        • Delete my own posts
                        • Delete my own threads
                        • Rate post

                        2000-2012 ASPPlayground.NET Forum Version 3.9