Tim's
on the right track with nbrbutil (which I hadn't seen before and isn't exactly
well-documented), but my particular media ID is still stubborn...
[135]
nbrbutil -cleandump | grep 000680
532491
DUP-READ 000680
[136]
nbrbutil -cancel 532491
No
request with ID {00000000-7F7F-0DB0-7F7F-12F00000002F} found
[137]
nbrbutil -cleandump | grep 000680
532491
DUP-READ 000680
[139]
nbrbutil -listOrphanedMedia
No
orphaned media
[140]
nbrbutil -releaseMedia 000680
No
allocation for mediaID 000680 found
But
actually reading the -cleandump (which, btw, isn't mentioned in the -help
output; cute!) output in context, I see:
MDS
allocations:
AllocationKey
JobType MediaID DriveName
MasterServer
MediaServer
================================================================================
532491
DUP-READ 000680
phlmaster
phlmaster
[...]
Oh,
so I guess that means we want this then:
[143]
nbrbutil -releaseMDS 532491
[144]
nbrbutil -cleandump | grep 000680
And
FINALLY:
[145]
bpmedia -unfreeze -m 000680
[146]
vmchange -m 000680 -d 'BAD MEDIA'
Thanks,
Tim!
Tim
also suggested checking bpimagelist's -stl_incomplete and -stl_complete flags,
but I already knew that state from nbstlutil, unless I'm mistaken (and -cancel
there, in any case, forcibly set the state to complete for those images.)
--
Gabriel
Rosenkoetter
Radian
Group Inc, Senior Systems Engineer
gabriel.rosenkoetter AT radian DOT biz,
215 231 1556
From: Tim Hoke
[mailto:thoke AT northpeak DOT org]
Sent: Monday, May 11, 2009 1:47 PM
To: Rosenkoetter, Gabriel
Cc: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] What to do about media stuck in the media DB
(plausibly due to storage lifecycle policies)
(Reply to all this time)...
Any allocations for that media id?
nbrbutil -cleandump (if cleandump doesn't work, just use -dump - this was an
updated option, but i don't recall the version).
>From there, you can use other options on nbrbutil to release that given
allocation (just be sure that there really aren't any other jobs waiting for
the resource)
-Tim
On Mon, May 11, 2009 at 12:23 PM, Rosenkoetter, Gabriel <Gabriel.Rosenkoetter AT radian DOT biz>
wrote:
PS, yes, there was still a duplication job active looking
for media ID 000680. After cancelling that job, however, I am still unable to
nuke these images (nbstlutil list is still blank for the media ID).
-----Original Message-----
From: Rosenkoetter, Gabriel
Sent: Monday, May 11, 2009 1:03 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: What to do about media stuck in the media DB (plausibly due to storage
lifecycle policies)
There is a question down at the bottom of this (how do I actually remove a
media ID that I know is unuseable and doesn't hold an unexpired images from the
media DB?), but a bit on what I've done so far:
I have a very old piece of 9940 media:
[97] vmquery -m 000680
================================================================================
media ID: 000680
media type: 1/2" cartridge tape 2
(14)
barcode: 000680
media description: Added by Media Manager
volume pool: 00_NetBackup (11)
robot type: NONE - Not Robotic (0)
volume group: ---
vault name: ---
vault sent date: ---
vault return date: ---
vault slot: ---
vault session id: ---
vault container id: -
created: Tue Feb 22 15:47:00
2005
assigned: Sat Apr 18 13:00:02
2009
last mounted: Sun Apr 26 21:15:10 2009
first mount: Fri May 20 17:27:16 2005
expiration date: ---
number of mounts: 626
max mounts allowed: ---
status: 0x0
================================================================================
It was frozen due to write errors (and periodically throws read errors as well)
and, given its age and mount count, I concluded that it was time to pull it out
of service after my SLPs had managed to get everything they could off of it and
the images on it had expired, which they now have:
[98] bpmedialist -m 000680
Server Host = phlmaster
id rl images allocated
last updated density kbytes restores
vimages expiration
last read <------- STATUS ------->
--------------------------------------------------------------------------------
000680 1 40 04/18/2009 13:00 04/20/2009 20:04
hcart2 38214373 0
MPX 05/04/2009 20:04
04/19/2009 10:59 EXPIRED FROZEN
It is, however, apparently locked in the media DB (because there's an
unreadable image and SLPs are still trying to duplicate those images, I
thought):
[96] bpmedia -unfreeze -m 000680
requested media id is in use, cannot process request
root@pa2ibks2:/usr/openv/netbackup/vault/sessions/P472B
[99] bpexpdate -m 000680 -d 0
Are you SURE you want to delete 000680 y/n (n)? y
requested media id is in use, cannot process request
Here are the lifecycles that are trying to read that tape (ignore what the man
page says about nbstlutil list's output being "just for support",
it's plenty readable):
[124] nbstlutil list -U -mediaid 000680
Image:
Master Server : phlmaster
Backup ID :
pa02qrin02_1240271378
Client :
pa02qrin02
Backup Time : 1240271378
(Mon Apr 20 19:49:38 2009)
Policy :
VM_Instances
Client Type : 13
Schedule Type : 4
Storage Lifecycle Policy : Cumulatives
Storage Lifecycle State : 2 (IN_PROCESS)
Time In Process : 1240272689 (Mon Apr
20 20:11:29 2009)
Data Classification ID : (none specified)
Copy:
Master Server : phlmaster
Backup ID : pa02qrin02_1240271378
Copy Number : 1
Expire Time : 2147483647 (Mon Jan 18
22:14:07 2038)
Try To Keep Time : 1241480978 (Mon May 4 19:49:38
2009)
Residence : PHL-9940b
Duplication Status : 3 (COMPLETE)
Job ID : 0
Retention Type : 0 (FIXED)
MPX State : 1
Fragment:
Master Server : phlmaster
Backup ID : pa02qrin02_1240271378
Copy Number : 1
Fragment Number : 1
Resume Count : 0
Media ID : 000680
Media Server : phlmaster
Storage Server : (none specified)
Media Type : 2 (RMEDIA)
Media Sub-Type : 0 (DEFAULT)
Fragment State : 1 (ACTIVE)
Fragment Size : 0
Delete Header : 1
Fragment ID : 000680
[... - there are many fragments]
You can't cancel by media ID, only by lifecycle, destination, or backupid, the
last of which we want here, after which the list output is empty:
[132] nbstlutil cancel -backupid pa02qrin02_1240271378
[133] nbstlutil list -U -mediaid 000680
But I STILL can't unfreeze or expire the media:
[134] bpmedia -unfreeze -m 000680
requested media id is in use, cannot process request
[137] bpexpdate -m 000680 -d 0
Are you SURE you want to delete 000680 y/n (n)? y
requested media id is in use, cannot process request
Any thoughts?
--
Gabriel Rosenkoetter
Radian Group Inc, Senior Systems Engineer
gabriel.rosenkoetter AT radian DOT biz,
215 231 1556
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu