Jul 17 17:23:47.859830 ip-10-0-135-30 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.314552    2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319924    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319943    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319947    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319950    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319953    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319956    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319961    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:23:48.344398 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319964    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319967    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319970    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319973    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319976    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319979    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319982    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319985    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319987    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319990    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319993    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.319996    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320000    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320003    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320005    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320008    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320010    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320015    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320019    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320022    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:23:48.559262 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320025    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:23:48.398826 ip-10-0-135-30 systemd[1]: Started Kubernetes Kubelet.
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320028    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320032    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320036    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320039    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320042    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320045    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320047    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320051    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320055    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320058    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320060    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320063    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320066    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320069    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320072    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320074    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320077    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320080    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:23:48.566365 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320082    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320085    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320088    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320090    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320093    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320096    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320099    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320101    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320104    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320108    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320110    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320122    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320125    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320127    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320131    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320134    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320138    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320140    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320143    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320145    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:23:48.567218 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320148    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320150    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320153    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320156    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320158    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320162    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320165    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320167    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320171    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320174    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320176    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320180    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320183    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320185    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320188    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320191    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320193    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320196    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320199    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320203    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:23:48.567922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320206    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320209    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320211    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320214    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320216    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320219    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320222    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320226    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320232    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320235    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320237    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320240    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.320243    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321035    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321043    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321048    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321052    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321055    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321058    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321061    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:23:48.568555 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321064    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321067    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321070    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321072    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321075    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321077    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321080    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321083    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321086    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321088    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321091    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321094    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321097    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321100    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321102    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321105    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321107    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321110    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321113    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:23:48.569173 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321117    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321124    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321127    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321130    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321134    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321138    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321141    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321144    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321147    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321150    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321154    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321157    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321160    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321162    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321165    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321167    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321173    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321175    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321178    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:23:48.569784 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321180    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321183    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321185    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321188    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321191    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321193    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321196    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321199    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321201    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321204    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321206    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321209    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321211    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321214    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321218    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321221    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321224    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321226    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321229    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321231    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:23:48.570481 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321234    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321236    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321239    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321242    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321244    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321247    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321249    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321252    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321254    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321259    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321261    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321264    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321266    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321270    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321272    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321275    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321277    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321280    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321283    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321285    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:23:48.571089 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321288    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321291    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321294    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321296    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321298    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321301    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321305    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321307    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321310    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321313    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321315    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321318    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321321    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321323    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321736    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321741    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321744    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321747    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321750    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:23:48.571745 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321753    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321756    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321760    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321762    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321765    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321768    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321771    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321773    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321777    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321779    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321782    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321784    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321787    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321791    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321794    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321797    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321800    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321802    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321805    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:23:48.572511 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321808    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321812    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321814    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321817    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321819    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321822    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321825    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321827    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321830    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321833    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321835    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321838    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321840    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321843    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321845    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321849    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321852    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321855    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321857    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321860    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:23:48.573401 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321862    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321865    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321868    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321870    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321872    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321875    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321877    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321880    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321883    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321886    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321888    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321890    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321894    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321898    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321902    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321905    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321908    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321912    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321915    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321918    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:23:48.574304 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321921    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321923    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321925    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321928    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321931    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321933    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321936    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321940    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321942    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321946    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321949    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321952    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321956    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321958    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321961    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321963    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321966    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321968    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321971    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:23:48.575403 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321973    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321976    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321979    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321981    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321984    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321988    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321990    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321993    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321996    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.321998    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322001    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322004    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322006    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322009    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322012    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322014    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322095    2572 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322108    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322115    2572 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322119    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322125    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 17:23:48.576237 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322128    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322132    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322136    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322140    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322143    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322146    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322150    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322153    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322156    2572 flags.go:64] FLAG: --cgroup-root=""
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322159    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322162    2572 flags.go:64] FLAG: --client-ca-file=""
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322165    2572 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322168    2572 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322173    2572 flags.go:64] FLAG: --cluster-domain=""
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322176    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322179    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322184    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322188    2572 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322191    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322195    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322199    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322202    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322206    2572 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322208    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 17:23:48.577352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322212    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322215    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322219    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322223    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322226    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322228    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322231    2572 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322237    2572 flags.go:64] FLAG: --enable-server="true"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322240    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322245    2572 flags.go:64] FLAG: --event-burst="100"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322248    2572 flags.go:64] FLAG: --event-qps="50"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322251    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322254    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322257    2572 flags.go:64] FLAG: --eviction-hard=""
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322261    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322264    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322267    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322270    2572 flags.go:64] FLAG: --eviction-soft=""
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322273    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322276    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322279    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322282    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322285    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322288    2572 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322292    2572 flags.go:64] FLAG: --feature-gates=""
Jul 17 17:23:48.578461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322296    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322299    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322302    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322305    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322308    2572 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322311    2572 flags.go:64] FLAG: --help="false"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322314    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322317    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322320    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322323    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322326    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322329    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322332    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322335    2572 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322340    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322343    2572 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322346    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322349    2572 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322352    2572 flags.go:64] FLAG: --kube-reserved=""
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322355    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322358    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322361    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322364    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322367    2572 flags.go:64] FLAG: --lock-file=""
Jul 17 17:23:48.579567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322370    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322372    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322375    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322381    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322385    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322387    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322391    2572 flags.go:64] FLAG: --logging-format="text"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322395    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322398    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322401    2572 flags.go:64] FLAG: --manifest-url=""
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322404    2572 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322408    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322411    2572 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322415    2572 flags.go:64] FLAG: --max-pods="110"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322418    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322422    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322425    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322427    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322431    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322433    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322437    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322444    2572 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322449    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322453    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 17:23:48.580889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322456    2572 flags.go:64] FLAG: --pod-cidr=""
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322459    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322465    2572 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322468    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322471    2572 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322474    2572 flags.go:64] FLAG: --port="10250"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322478    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322481    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b54eb88e4534a45f"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322484    2572 flags.go:64] FLAG: --qos-reserved=""
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322504    2572 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322508    2572 flags.go:64] FLAG: --register-node="true"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322511    2572 flags.go:64] FLAG: --register-with-taints=""
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322514    2572 flags.go:64] FLAG: --registry-burst="10"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322517    2572 flags.go:64] FLAG: --registry-qps="5"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322520    2572 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322523    2572 flags.go:64] FLAG: --reserved-memory=""
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322529    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322532    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322535    2572 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322538    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322541    2572 flags.go:64] FLAG: --runonce="false"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322544    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322547    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322550    2572 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322553    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 17:23:48.582006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322556    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322559    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322562    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322565    2572 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322568    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322573    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322575    2572 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322579    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322582    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322585    2572 flags.go:64] FLAG: --system-cgroups=""
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322587    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322593    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322596    2572 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322599    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322603    2572 flags.go:64] FLAG: --tls-min-version=""
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322605    2572 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322608    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322611    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322614    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322617    2572 flags.go:64] FLAG: --v="2"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322621    2572 flags.go:64] FLAG: --version="false"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322625    2572 flags.go:64] FLAG: --vmodule=""
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322628    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.322634    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322757    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:23:48.586898 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322762    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322765    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322768    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322771    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322773    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322776    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322779    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322781    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322784    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322787    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322789    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322792    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322796    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322799    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322802    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322804    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322807    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322809    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322811    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322814    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:23:48.588206 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322817    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322819    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322822    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322824    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322827    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322829    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322832    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322834    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322837    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322840    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322843    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322846    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322849    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322851    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322854    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322856    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322859    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322862    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322865    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322867    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:23:48.728828 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322870    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322872    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322875    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322877    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322881    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322884    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322886    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322890    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322894    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322897    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322901    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322904    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322907    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322910    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322913    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322916    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322919    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322922    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322924    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:23:48.729922 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322927    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322930    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322933    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322937    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322940    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322944    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322947    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322949    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322952    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322954    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322957    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322959    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322962    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322965    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322967    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322970    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322972    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322977    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322979    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322982    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:23:48.730804 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322984    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322987    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322990    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322993    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322995    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.322998    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323000    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323003    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323005    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323008    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323010    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323012    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323015    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323019    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323022    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323026    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323030    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323032    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:23:48.731676 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.323036    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.323760    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.333612    2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.333636    2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.334275    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.334283    2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334350    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334356    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334359    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334362    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334366    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334369    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334372    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334375    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:23:48.732762 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334378    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334382    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334384    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334387    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334390    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334393    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334396    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334400    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334402    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334405    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334408    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334410    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334413    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334416    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334418    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334422    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334425    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334428    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334431    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334434    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:23:48.733416 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334436    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334439    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334441    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334445    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334447    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334450    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334452    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334455    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334458    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334461    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334463    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334466    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334468    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334471    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334474    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334477    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334480    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334482    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334502    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334507    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:23:48.734300 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334510    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334513    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334515    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334521    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334523    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334526    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334529    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334532    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334535    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334538    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334541    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334546    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334548    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334551    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334553    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334556    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334559    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334562    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334564    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334567    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:23:48.735185 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334569    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334572    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334575    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334577    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334580    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334583    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334586    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334588    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334591    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334595    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334599    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334602    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334606    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334609    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334611    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334617    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334621    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334624    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334628    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:23:48.736061 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334633    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334636    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334639    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334642    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334647    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334649    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334652    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334655    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334658    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334660    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334663    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334666    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.334672    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334795    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334801    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:23:48.736961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334804    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334807    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334810    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334813    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334816    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334819    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334822    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334825    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334828    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334831    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334834    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334837    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334840    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334843    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334847    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334849    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334852    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334855    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334857    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:23:48.737615 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334860    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334862    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334867    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334870    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334873    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334875    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334877    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334880    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334883    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334885    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334888    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334890    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334893    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334896    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334898    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334900    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334903    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334906    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334908    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334911    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:23:48.738501 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334913    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334916    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334919    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334921    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334925    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334928    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334931    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334935    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334938    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334940    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334943    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334946    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334948    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334952    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334957    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334960    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334963    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334966    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334969    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:23:48.739355 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334971    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334974    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334976    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334979    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334981    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334984    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334986    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334989    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334991    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334994    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334996    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.334999    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335001    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335004    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335006    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335009    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335011    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335014    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335017    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335019    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:23:48.740239 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335023    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335026    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335028    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335031    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335033    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335036    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335039    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335042    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335045    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335048    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335050    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335053    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335055    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335059    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335063    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335065    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335068    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335071    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:23:48.741124 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.335073    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.335078    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.335219    2572 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.340531    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.342178    2572 server.go:1019] "Starting client certificate rotation"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.342298    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.342348    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.345734    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.347052    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.353945    2572 log.go:25] "Validated CRI v1 runtime API"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.357431    2572 log.go:25] "Validated CRI v1 image API"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.361473    2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 17:23:48.742002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.364733    2572 fs.go:135] Filesystem UUIDs: map[67ceece7-c45c-495a-a73c-00d3281d79e4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d6e6cc5e-f184-4dfe-aa3b-8763c0e36128:/dev/nvme0n1p3]
Jul 17 17:23:48.742865 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.364754    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 17:23:48.743087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.371728    2572 manager.go:216] Machine: {Timestamp:2026-07-17 17:23:48.370201164 +0000 UTC m=+0.375557017 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095439 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25f0d64ef25a969b44ea0023fe84a4 SystemUUID:ec25f0d6-4ef2-5a96-9b44-ea0023fe84a4 BootID:a5c440c9-500e-463c-8695-58bcc80b0f74 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:72:65:40:ad:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:72:65:40:ad:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:8c:c8:5d:40:55 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 17:23:48.743087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.371871    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 17:23:48.743087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.371987    2572 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 17:23:48.743087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.373965    2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 17:23:48.743087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.374012    2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-30.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 17:23:48.743087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.374849    2572 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 17:23:48.743087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.374866    2572 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 17:23:48.743087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.374876    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.376360    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.378075    2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.379757    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.381583    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.381762    2572 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.381807    2572 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.381823    2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.381845    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.381861    2572 kubelet.go:398] "Adding apiserver pod source"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.381876    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.383075    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.383098    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.387755    2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.390610    2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.390644    2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392011    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392026    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392032    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392037    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392049    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392057    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392070    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392076    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392084    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392089    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392098    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.392104    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.393877    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.393890    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.394040    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.394173    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.397359    2572 server.go:1268] "Started kubelet"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.397441    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.397533    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.397617    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.400053    2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.400089    2572 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.404820    2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 17:23:48.743737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.404839    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-30.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.404758    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-30.ec2.internal.18c3239b8f250bfe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-30.ec2.internal,UID:ip-10-0-135-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-30.ec2.internal,},FirstTimestamp:2026-07-17 17:23:48.397321214 +0000 UTC m=+0.402677060,LastTimestamp:2026-07-17 17:23:48.397321214 +0000 UTC m=+0.402677060,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-30.ec2.internal,}"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.410130    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.410241    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.410171    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.410889    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411022    2572 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411041    2572 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411166    2572 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411174    2572 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411647    2572 factory.go:153] Registering CRI-O factory
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411666    2572 factory.go:223] Registration of the crio container factory successfully
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411734    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411744    2572 factory.go:55] Registering systemd factory
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411752    2572 factory.go:223] Registration of the systemd container factory successfully
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411772    2572 factory.go:103] Registering Raw factory
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.411785    2572 manager.go:1196] Started watching for new ooms in manager
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.412204    2572 manager.go:349] Starting recovery of all containers
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.413319    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.413906    2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:48.417088    2572 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service/memory.swap.max": read /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service/memory.swap.max: no such device
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.422832    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bgf6x"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.425667    2572 manager.go:354] Recovery completed
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.428915    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bgf6x"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.431013    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.433313    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.433347    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.433360    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.434049    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.434064    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.434072    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.434083    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 17:23:48.745528 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.434775    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.434803    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.434817    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.435159    2572 policy_none.go:49] "None policy: Start"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.435176    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.435190    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.436151    2572 policy_none.go:47] "Start"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.484302    2572 manager.go:337] "Starting Device Plugin manager"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.484343    2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.484355    2572 server.go:87] "Starting device plugin registration server"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.484703    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.484717    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.484937    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.485034    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.485044    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.485429    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.485471    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.507645    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.508987    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.509030    2572 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.509061    2572 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.509102    2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.511509    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.585528    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.586629    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.586654    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.586665    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.586691    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.595435    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.595466    2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-30.ec2.internal\": node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.609382    2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal"]
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.609461    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.610548    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.610583    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.610596    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.611833    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.611938    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.611967    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:23:48.749189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.612891    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.612923    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.612937    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.614471    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.614517    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.614748    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.614774    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.614789    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.615211    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.615230    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.615240    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.624483    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.654632    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-30.ec2.internal\" not found" node="ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.659277    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-30.ec2.internal\" not found" node="ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.713629    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6fa073bc5f1e1fe550b2b25522ade8d4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal\" (UID: \"6fa073bc5f1e1fe550b2b25522ade8d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.713660    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fa073bc5f1e1fe550b2b25522ade8d4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal\" (UID: \"6fa073bc5f1e1fe550b2b25522ade8d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.713683    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73ecd14a99b2d4ac78d23a5262bad321-config\") pod \"kube-apiserver-proxy-ip-10-0-135-30.ec2.internal\" (UID: \"73ecd14a99b2d4ac78d23a5262bad321\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.750723 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.724725    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:48.923586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.814031    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6fa073bc5f1e1fe550b2b25522ade8d4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal\" (UID: \"6fa073bc5f1e1fe550b2b25522ade8d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.923586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.814077    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fa073bc5f1e1fe550b2b25522ade8d4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal\" (UID: \"6fa073bc5f1e1fe550b2b25522ade8d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.923586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.814096    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73ecd14a99b2d4ac78d23a5262bad321-config\") pod \"kube-apiserver-proxy-ip-10-0-135-30.ec2.internal\" (UID: \"73ecd14a99b2d4ac78d23a5262bad321\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.923586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.814126    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73ecd14a99b2d4ac78d23a5262bad321-config\") pod \"kube-apiserver-proxy-ip-10-0-135-30.ec2.internal\" (UID: \"73ecd14a99b2d4ac78d23a5262bad321\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.923586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.814127    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6fa073bc5f1e1fe550b2b25522ade8d4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal\" (UID: \"6fa073bc5f1e1fe550b2b25522ade8d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.923586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.814145    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fa073bc5f1e1fe550b2b25522ade8d4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal\" (UID: \"6fa073bc5f1e1fe550b2b25522ade8d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:48.923586 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.825144    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:48.926084 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:48.925684    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:49.092206 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.958275    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:49.092206 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:48.962411    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:49.092206 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.025939    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:49.297374 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.126998    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:49.297374 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.228108    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.328262    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-30.ec2.internal\" not found"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.342674    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.342846    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.359900    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.382118    2572 apiserver.go:52] "Watching apiserver"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.392826    2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.394032    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7","openshift-cluster-node-tuning-operator/tuned-tf4tf","openshift-dns/node-resolver-l2pbp","openshift-multus/multus-2xkgl","openshift-multus/network-metrics-daemon-zvc9x","openshift-ovn-kubernetes/ovnkube-node-bbmxm","kube-system/global-pull-secret-syncer-p8nmp","openshift-image-registry/node-ca-l9hsk","openshift-multus/multus-additional-cni-plugins-qgc79","openshift-network-diagnostics/network-check-target-2cr94","openshift-network-operator/iptables-alerter-8hmdb","kube-system/konnectivity-agent-vbj8d"]
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.395594    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.397238    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.398500    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.398664    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.406760    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.407062    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.407086    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.407392    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4szdp\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.407411    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vdvs5\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.407545    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.407910    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.409160    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.409257    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.410434    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.410458    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.411157    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.411507    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.411546    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.411791    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 17:23:49.422046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.411839    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.411895    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qvf5v\""
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.412021    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.412054    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8tj6n\""
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.412061    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.412504    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.412562    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.413915    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.413914    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.416490    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.417821    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.418879    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-etc-openvswitch\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.418913    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-run-ovn-kubernetes\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.418948    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-device-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.418994    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvw67\" (UniqueName: \"kubernetes.io/projected/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-kube-api-access-hvw67\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419026    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-sys\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419068    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/12657b07-032c-4033-88ac-46e27583c859-hosts-file\") pod \"node-resolver-l2pbp\" (UID: \"12657b07-032c-4033-88ac-46e27583c859\") " pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419090    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-etc-kubernetes\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419121    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-run-systemd\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.423980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419143    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-kubernetes\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419166    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-sysctl-conf\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419185    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e09f9cb-3818-45a0-85ab-586d624a8935-tmp\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419203    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-log-socket\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419237    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-cni-netd\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419266    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-system-cni-dir\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419289    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-hostroot\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419314    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-var-lib-cni-bin\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419337    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-systemd-units\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419359    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419381    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-etc-selinux\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419403    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-modprobe-d\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419422    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-host\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419446    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-tuned\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.425602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419466    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-run-k8s-cni-cncf-io\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.426531 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419488    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00473330-68a9-4b11-8a60-732329b1891a-ovnkube-script-lib\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.426531 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419509    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-socket-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.426531 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419542    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-systemd\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.426531 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419562    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00473330-68a9-4b11-8a60-732329b1891a-env-overrides\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.426531 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419583    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-cnibin\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.426531 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419621    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-run-ovn\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.426531 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419644    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-cni-bin\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.434448 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.434074    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 17:18:49 +0000 UTC" deadline="2028-02-15 00:52:20.297981653 +0000 UTC"
Jul 17 17:23:49.434448 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.434115    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13855h28m30.863870422s"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.419668    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442460    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-sysctl-d\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442493    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-registration-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442516    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-lib-modules\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442540    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lz72\" (UniqueName: \"kubernetes.io/projected/12657b07-032c-4033-88ac-46e27583c859-kube-api-access-9lz72\") pod \"node-resolver-l2pbp\" (UID: \"12657b07-032c-4033-88ac-46e27583c859\") " pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.415177    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442582    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-var-lib-kubelet\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442632    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-os-release\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442660    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd33444c-515f-4f4d-a793-933561aee26d-cni-binary-copy\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442682    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-run-netns\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442703    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-run-openvswitch\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442744    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00473330-68a9-4b11-8a60-732329b1891a-ovn-node-metrics-cert\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442767    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-var-lib-kubelet\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.554268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12657b07-032c-4033-88ac-46e27583c859-tmp-dir\") pod \"node-resolver-l2pbp\" (UID: \"12657b07-032c-4033-88ac-46e27583c859\") " pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442809    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-var-lib-cni-multus\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442830    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-run-netns\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442831    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b664h\""
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.442911    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.443075    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.443091    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.443385    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444709    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-kubelet\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.444706    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444734    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444751    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-node-log\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444783    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-run\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444789    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444829    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5f42\" (UniqueName: \"kubernetes.io/projected/4e09f9cb-3818-45a0-85ab-586d624a8935-kube-api-access-w5f42\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444862    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-multus-socket-dir-parent\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444889    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd33444c-515f-4f4d-a793-933561aee26d-multus-daemon-config\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.556113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444912    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-sysconfig\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444928    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-run-multus-certs\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444944    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ddm4\" (UniqueName: \"kubernetes.io/projected/bd33444c-515f-4f4d-a793-933561aee26d-kube-api-access-9ddm4\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444967    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpqh5\" (UniqueName: \"kubernetes.io/projected/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-kube-api-access-qpqh5\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.444996    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-var-lib-openvswitch\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445058    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445074    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445083    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-multus-conf-dir\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445113    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-slash\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445145    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-sys-fs\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445168    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00473330-68a9-4b11-8a60-732329b1891a-ovnkube-config\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445191    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snj7k\" (UniqueName: \"kubernetes.io/projected/00473330-68a9-4b11-8a60-732329b1891a-kube-api-access-snj7k\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445236    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-multus-cni-dir\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445714    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445720    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5dwm4\""
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445730    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445752    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 17:23:49.557982 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445779    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445785    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445720    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445836    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.445760    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wxwbd\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.447002    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal"]
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.447105    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.448289    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.448292    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7fz46\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.449617    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9d4rp\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.449730    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.449813    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.450872    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.453777    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2prlj"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.457800    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.458113    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal"]
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.458872    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2prlj"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.511859    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.545798    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-cnibin\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.545828    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aaf00cc8-00f5-4664-8764-81e49ed6d9aa-host-slash\") pod \"iptables-alerter-8hmdb\" (UID: \"aaf00cc8-00f5-4664-8764-81e49ed6d9aa\") " pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.545849    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-run-ovn\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.545900    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-cni-bin\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.545933    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:49.559167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.545953    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-cni-bin\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.545955    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aaf00cc8-00f5-4664-8764-81e49ed6d9aa-iptables-alerter-script\") pod \"iptables-alerter-8hmdb\" (UID: \"aaf00cc8-00f5-4664-8764-81e49ed6d9aa\") " pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.545987    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-sysctl-d\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj9x7\" (UniqueName: \"kubernetes.io/projected/1fb7e912-6649-48cc-88c7-989f172f937c-kube-api-access-pj9x7\") pod \"node-ca-l9hsk\" (UID: \"1fb7e912-6649-48cc-88c7-989f172f937c\") " pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546031    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-registration-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.545904    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-run-ovn\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546058    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-lib-modules\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546081    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lz72\" (UniqueName: \"kubernetes.io/projected/12657b07-032c-4033-88ac-46e27583c859-kube-api-access-9lz72\") pod \"node-resolver-l2pbp\" (UID: \"12657b07-032c-4033-88ac-46e27583c859\") " pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546103    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-var-lib-kubelet\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546114    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-registration-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546124    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-os-release\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546148    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd33444c-515f-4f4d-a793-933561aee26d-cni-binary-copy\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546164    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-sysctl-d\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546173    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-run-netns\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546204    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-os-release\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.560615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546164    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-lib-modules\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546209    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-var-lib-kubelet\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546208    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-run-openvswitch\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546209    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-run-netns\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546242    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-run-openvswitch\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546311    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00473330-68a9-4b11-8a60-732329b1891a-ovn-node-metrics-cert\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546353    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-var-lib-kubelet\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546373    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12657b07-032c-4033-88ac-46e27583c859-tmp-dir\") pod \"node-resolver-l2pbp\" (UID: \"12657b07-032c-4033-88ac-46e27583c859\") " pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546395    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-var-lib-cni-multus\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546417    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-var-lib-kubelet\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546419    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-run-netns\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546448    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-run-netns\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546459    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-kubelet\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546475    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-node-log\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546482    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-var-lib-cni-multus\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.561559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546515    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-run\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546525    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-kubelet\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546535    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-node-log\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546544    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5f42\" (UniqueName: \"kubernetes.io/projected/4e09f9cb-3818-45a0-85ab-586d624a8935-kube-api-access-w5f42\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546572    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-multus-socket-dir-parent\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546601    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd33444c-515f-4f4d-a793-933561aee26d-multus-daemon-config\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.546584    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546650    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-run\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546656    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-multus-socket-dir-parent\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.546721    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs podName:8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d nodeName:}" failed. No retries permitted until 2026-07-17 17:23:50.046674396 +0000 UTC m=+2.052030255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs") pod "network-metrics-daemon-zvc9x" (UID: "8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546743    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-sysconfig\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546771    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-run-multus-certs\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546799    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ddm4\" (UniqueName: \"kubernetes.io/projected/bd33444c-515f-4f4d-a793-933561aee26d-kube-api-access-9ddm4\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546820    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-sysconfig\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.562480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546831    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpqh5\" (UniqueName: \"kubernetes.io/projected/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-kube-api-access-qpqh5\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546841    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-run-multus-certs\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546860    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-var-lib-openvswitch\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546781    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd33444c-515f-4f4d-a793-933561aee26d-cni-binary-copy\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546886    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546946    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.546946    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547016    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-var-lib-openvswitch\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547057    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-multus-conf-dir\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547077    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-multus-conf-dir\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547085    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-system-cni-dir\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547129    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-os-release\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547141    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd33444c-515f-4f4d-a793-933561aee26d-multus-daemon-config\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547181    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.564999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547213    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e32df813-b6ae-45ff-9b71-82510269769b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547243    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-slash\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547239    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12657b07-032c-4033-88ac-46e27583c859-tmp-dir\") pod \"node-resolver-l2pbp\" (UID: \"12657b07-032c-4033-88ac-46e27583c859\") " pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547290    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-sys-fs\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547310    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-slash\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547332    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547355    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-sys-fs\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547364    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed5123f0-04f4-45e5-8098-6827fe05c426-konnectivity-ca\") pod \"konnectivity-agent-vbj8d\" (UID: \"ed5123f0-04f4-45e5-8098-6827fe05c426\") " pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547392    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00473330-68a9-4b11-8a60-732329b1891a-ovnkube-config\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547431    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snj7k\" (UniqueName: \"kubernetes.io/projected/00473330-68a9-4b11-8a60-732329b1891a-kube-api-access-snj7k\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547538    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-multus-cni-dir\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547573    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e32df813-b6ae-45ff-9b71-82510269769b-cni-binary-copy\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547601    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed5123f0-04f4-45e5-8098-6827fe05c426-agent-certs\") pod \"konnectivity-agent-vbj8d\" (UID: \"ed5123f0-04f4-45e5-8098-6827fe05c426\") " pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547630    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-etc-openvswitch\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.565947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547655    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-run-ovn-kubernetes\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547680    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-device-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547729    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-etc-openvswitch\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547771    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-run-ovn-kubernetes\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547807    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvw67\" (UniqueName: \"kubernetes.io/projected/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-kube-api-access-hvw67\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547830    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-device-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547828    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-multus-cni-dir\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547837    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-sys\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547874    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/12657b07-032c-4033-88ac-46e27583c859-hosts-file\") pod \"node-resolver-l2pbp\" (UID: \"12657b07-032c-4033-88ac-46e27583c859\") " pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547881    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-sys\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547899    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-etc-kubernetes\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547921    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00473330-68a9-4b11-8a60-732329b1891a-ovnkube-config\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547925    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-run-systemd\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547930    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/12657b07-032c-4033-88ac-46e27583c859-hosts-file\") pod \"node-resolver-l2pbp\" (UID: \"12657b07-032c-4033-88ac-46e27583c859\") " pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547952    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-kubernetes\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.566669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547971    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-etc-kubernetes\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547976    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-run-systemd\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.547981    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-sysctl-conf\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548006    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e09f9cb-3818-45a0-85ab-586d624a8935-tmp\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548014    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-kubernetes\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548039    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e32df813-b6ae-45ff-9b71-82510269769b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548062    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkqrm\" (UniqueName: \"kubernetes.io/projected/aaf00cc8-00f5-4664-8764-81e49ed6d9aa-kube-api-access-kkqrm\") pod \"iptables-alerter-8hmdb\" (UID: \"aaf00cc8-00f5-4664-8764-81e49ed6d9aa\") " pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548080    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-log-socket\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548138    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-cni-netd\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548152    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-sysctl-conf\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548159    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-system-cni-dir\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548182    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-hostroot\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548185    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-log-socket\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548210    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-var-lib-cni-bin\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548215    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-system-cni-dir\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.567827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548225    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-cni-netd\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548237    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-systemd-units\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548264    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548266    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-var-lib-cni-bin\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548233    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-hostroot\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548282    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-systemd-units\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548301    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-etc-selinux\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548339    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00473330-68a9-4b11-8a60-732329b1891a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548357    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-modprobe-d\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548395    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-host\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548410    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-etc-selinux\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548444    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-host\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548456    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-tuned\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548479    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1fb7e912-6649-48cc-88c7-989f172f937c-serviceca\") pod \"node-ca-l9hsk\" (UID: \"1fb7e912-6649-48cc-88c7-989f172f937c\") " pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.569419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548511    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-modprobe-d\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548530    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-run-k8s-cni-cncf-io\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548567    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00473330-68a9-4b11-8a60-732329b1891a-ovnkube-script-lib\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-socket-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548633    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-systemd\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548630    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-host-run-k8s-cni-cncf-io\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548652    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf8kt\" (UniqueName: \"kubernetes.io/projected/e32df813-b6ae-45ff-9b71-82510269769b-kube-api-access-kf8kt\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548685    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00473330-68a9-4b11-8a60-732329b1891a-env-overrides\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548741    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-systemd\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548758    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-socket-dir\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548776    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-cnibin\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548806    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-kubelet-config\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548828    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-dbus\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548857    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:49.570362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548884    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd33444c-515f-4f4d-a793-933561aee26d-cnibin\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.548883    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fb7e912-6649-48cc-88c7-989f172f937c-host\") pod \"node-ca-l9hsk\" (UID: \"1fb7e912-6649-48cc-88c7-989f172f937c\") " pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.549127    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00473330-68a9-4b11-8a60-732329b1891a-ovnkube-script-lib\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.549133    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00473330-68a9-4b11-8a60-732329b1891a-env-overrides\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.551089    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e09f9cb-3818-45a0-85ab-586d624a8935-tmp\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.551250    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00473330-68a9-4b11-8a60-732329b1891a-ovn-node-metrics-cert\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.551286    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e09f9cb-3818-45a0-85ab-586d624a8935-etc-tuned\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.561953    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.565204    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ddm4\" (UniqueName: \"kubernetes.io/projected/bd33444c-515f-4f4d-a793-933561aee26d-kube-api-access-9ddm4\") pod \"multus-2xkgl\" (UID: \"bd33444c-515f-4f4d-a793-933561aee26d\") " pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.565999    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snj7k\" (UniqueName: \"kubernetes.io/projected/00473330-68a9-4b11-8a60-732329b1891a-kube-api-access-snj7k\") pod \"ovnkube-node-bbmxm\" (UID: \"00473330-68a9-4b11-8a60-732329b1891a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.566696    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5f42\" (UniqueName: \"kubernetes.io/projected/4e09f9cb-3818-45a0-85ab-586d624a8935-kube-api-access-w5f42\") pod \"tuned-tf4tf\" (UID: \"4e09f9cb-3818-45a0-85ab-586d624a8935\") " pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.566715    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lz72\" (UniqueName: \"kubernetes.io/projected/12657b07-032c-4033-88ac-46e27583c859-kube-api-access-9lz72\") pod \"node-resolver-l2pbp\" (UID: \"12657b07-032c-4033-88ac-46e27583c859\") " pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.566737    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpqh5\" (UniqueName: \"kubernetes.io/projected/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-kube-api-access-qpqh5\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:49.571579 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.568602    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvw67\" (UniqueName: \"kubernetes.io/projected/363a912a-bf5d-4ab7-9c2b-21698b1c4f1e-kube-api-access-hvw67\") pod \"aws-ebs-csi-driver-node-9fnb7\" (UID: \"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649277    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf8kt\" (UniqueName: \"kubernetes.io/projected/e32df813-b6ae-45ff-9b71-82510269769b-kube-api-access-kf8kt\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649323    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-kubelet-config\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649347    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-dbus\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649371    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649395    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fb7e912-6649-48cc-88c7-989f172f937c-host\") pod \"node-ca-l9hsk\" (UID: \"1fb7e912-6649-48cc-88c7-989f172f937c\") " pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649450    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-kubelet-config\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649506    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-dbus\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649527    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-cnibin\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.649542    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649559    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aaf00cc8-00f5-4664-8764-81e49ed6d9aa-host-slash\") pod \"iptables-alerter-8hmdb\" (UID: \"aaf00cc8-00f5-4664-8764-81e49ed6d9aa\") " pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649576    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-cnibin\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649521    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fb7e912-6649-48cc-88c7-989f172f937c-host\") pod \"node-ca-l9hsk\" (UID: \"1fb7e912-6649-48cc-88c7-989f172f937c\") " pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.649620    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret podName:7b5d7d3c-465d-40c3-8b32-d303103c7dd9 nodeName:}" failed. No retries permitted until 2026-07-17 17:23:50.1495941 +0000 UTC m=+2.154949949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret") pod "global-pull-secret-syncer-p8nmp" (UID: "7b5d7d3c-465d-40c3-8b32-d303103c7dd9") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:49.702101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649653    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aaf00cc8-00f5-4664-8764-81e49ed6d9aa-host-slash\") pod \"iptables-alerter-8hmdb\" (UID: \"aaf00cc8-00f5-4664-8764-81e49ed6d9aa\") " pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649693    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aaf00cc8-00f5-4664-8764-81e49ed6d9aa-iptables-alerter-script\") pod \"iptables-alerter-8hmdb\" (UID: \"aaf00cc8-00f5-4664-8764-81e49ed6d9aa\") " pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.649727    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj9x7\" (UniqueName: \"kubernetes.io/projected/1fb7e912-6649-48cc-88c7-989f172f937c-kube-api-access-pj9x7\") pod \"node-ca-l9hsk\" (UID: \"1fb7e912-6649-48cc-88c7-989f172f937c\") " pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.650033    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-system-cni-dir\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.650110    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-os-release\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.650140    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.650169    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e32df813-b6ae-45ff-9b71-82510269769b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.650561    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.650621    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed5123f0-04f4-45e5-8098-6827fe05c426-konnectivity-ca\") pod \"konnectivity-agent-vbj8d\" (UID: \"ed5123f0-04f4-45e5-8098-6827fe05c426\") " pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.650720    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-os-release\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.650725    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e32df813-b6ae-45ff-9b71-82510269769b-cni-binary-copy\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.650783    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aaf00cc8-00f5-4664-8764-81e49ed6d9aa-iptables-alerter-script\") pod \"iptables-alerter-8hmdb\" (UID: \"aaf00cc8-00f5-4664-8764-81e49ed6d9aa\") " pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.651166    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.651391    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed5123f0-04f4-45e5-8098-6827fe05c426-konnectivity-ca\") pod \"konnectivity-agent-vbj8d\" (UID: \"ed5123f0-04f4-45e5-8098-6827fe05c426\") " pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:23:49.703953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.651724    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e32df813-b6ae-45ff-9b71-82510269769b-cni-binary-copy\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.651733    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e32df813-b6ae-45ff-9b71-82510269769b-system-cni-dir\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.651782    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed5123f0-04f4-45e5-8098-6827fe05c426-agent-certs\") pod \"konnectivity-agent-vbj8d\" (UID: \"ed5123f0-04f4-45e5-8098-6827fe05c426\") " pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.651954    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e32df813-b6ae-45ff-9b71-82510269769b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.651999    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkqrm\" (UniqueName: \"kubernetes.io/projected/aaf00cc8-00f5-4664-8764-81e49ed6d9aa-kube-api-access-kkqrm\") pod \"iptables-alerter-8hmdb\" (UID: \"aaf00cc8-00f5-4664-8764-81e49ed6d9aa\") " pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.652048    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1fb7e912-6649-48cc-88c7-989f172f937c-serviceca\") pod \"node-ca-l9hsk\" (UID: \"1fb7e912-6649-48cc-88c7-989f172f937c\") " pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.652094    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e32df813-b6ae-45ff-9b71-82510269769b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.654722    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e32df813-b6ae-45ff-9b71-82510269769b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.655298    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1fb7e912-6649-48cc-88c7-989f172f937c-serviceca\") pod \"node-ca-l9hsk\" (UID: \"1fb7e912-6649-48cc-88c7-989f172f937c\") " pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.655406    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed5123f0-04f4-45e5-8098-6827fe05c426-agent-certs\") pod \"konnectivity-agent-vbj8d\" (UID: \"ed5123f0-04f4-45e5-8098-6827fe05c426\") " pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.657454    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.659060    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj9x7\" (UniqueName: \"kubernetes.io/projected/1fb7e912-6649-48cc-88c7-989f172f937c-kube-api-access-pj9x7\") pod \"node-ca-l9hsk\" (UID: \"1fb7e912-6649-48cc-88c7-989f172f937c\") " pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.659630    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf8kt\" (UniqueName: \"kubernetes.io/projected/e32df813-b6ae-45ff-9b71-82510269769b-kube-api-access-kf8kt\") pod \"multus-additional-cni-plugins-qgc79\" (UID: \"e32df813-b6ae-45ff-9b71-82510269769b\") " pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.660836    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.660855    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:23:49.704931 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.660869    2572 projected.go:196] Error preparing data for projected volume kube-api-access-cvzj8 for pod openshift-network-diagnostics/network-check-target-2cr94: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:49.706268 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:49.660945    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8 podName:1b9381cd-6d3e-4100-bd3c-d159d489f3d9 nodeName:}" failed. No retries permitted until 2026-07-17 17:23:50.160924253 +0000 UTC m=+2.166280105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cvzj8" (UniqueName: "kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8") pod "network-check-target-2cr94" (UID: "1b9381cd-6d3e-4100-bd3c-d159d489f3d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:49.706268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.663080    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkqrm\" (UniqueName: \"kubernetes.io/projected/aaf00cc8-00f5-4664-8764-81e49ed6d9aa-kube-api-access-kkqrm\") pod \"iptables-alerter-8hmdb\" (UID: \"aaf00cc8-00f5-4664-8764-81e49ed6d9aa\") " pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:49.706268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.672697    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:23:49.706268 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:49.685100    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ecd14a99b2d4ac78d23a5262bad321.slice/crio-6f911bbc0490441174f9064b342ca71f1c5ca03a3fe25c3dfa49c18dcdc77c83 WatchSource:0}: Error finding container 6f911bbc0490441174f9064b342ca71f1c5ca03a3fe25c3dfa49c18dcdc77c83: Status 404 returned error can't find the container with id 6f911bbc0490441174f9064b342ca71f1c5ca03a3fe25c3dfa49c18dcdc77c83
Jul 17 17:23:49.706268 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:49.685653    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5123f0_04f4_45e5_8098_6827fe05c426.slice/crio-7093f266e6672ce451523f06a154f13a2bc460da8124492743bd0f7c5519f9ef WatchSource:0}: Error finding container 7093f266e6672ce451523f06a154f13a2bc460da8124492743bd0f7c5519f9ef: Status 404 returned error can't find the container with id 7093f266e6672ce451523f06a154f13a2bc460da8124492743bd0f7c5519f9ef
Jul 17 17:23:49.706268 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:49.686508    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa073bc5f1e1fe550b2b25522ade8d4.slice/crio-40ca81c6c7c9e374d2e538376feb8931ec5c390f6c65b05d6c9203b3a96f6623 WatchSource:0}: Error finding container 40ca81c6c7c9e374d2e538376feb8931ec5c390f6c65b05d6c9203b3a96f6623: Status 404 returned error can't find the container with id 40ca81c6c7c9e374d2e538376feb8931ec5c390f6c65b05d6c9203b3a96f6623
Jul 17 17:23:49.706268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.691267    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 17:23:49.807210 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.765585    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7"
Jul 17 17:23:49.807210 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.800910    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tf4tf"
Jul 17 17:23:49.897310 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.824152    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2xkgl"
Jul 17 17:23:49.897310 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.840352    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l2pbp"
Jul 17 17:23:49.897310 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.860416    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:23:49.897310 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:49.863296    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12657b07_032c_4033_88ac_46e27583c859.slice/crio-019d316c9464d180c5f1be6bb9ba4368e8100cd487c6279f45cc963d2c38887e WatchSource:0}: Error finding container 019d316c9464d180c5f1be6bb9ba4368e8100cd487c6279f45cc963d2c38887e: Status 404 returned error can't find the container with id 019d316c9464d180c5f1be6bb9ba4368e8100cd487c6279f45cc963d2c38887e
Jul 17 17:23:49.897310 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:49.884087    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00473330_68a9_4b11_8a60_732329b1891a.slice/crio-3a9f7a9376bc37d3685749afa87e5abaecca34d5af776028df075eba47796857 WatchSource:0}: Error finding container 3a9f7a9376bc37d3685749afa87e5abaecca34d5af776028df075eba47796857: Status 404 returned error can't find the container with id 3a9f7a9376bc37d3685749afa87e5abaecca34d5af776028df075eba47796857
Jul 17 17:23:49.908872 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.908281    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l9hsk"
Jul 17 17:23:50.058813 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:49.917937    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fb7e912_6649_48cc_88c7_989f172f937c.slice/crio-45431d5a231b5d71ef183a355715cb43c3f9b5d92e68ab3b98db021ea6e86274 WatchSource:0}: Error finding container 45431d5a231b5d71ef183a355715cb43c3f9b5d92e68ab3b98db021ea6e86274: Status 404 returned error can't find the container with id 45431d5a231b5d71ef183a355715cb43c3f9b5d92e68ab3b98db021ea6e86274
Jul 17 17:23:50.058813 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.929463    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8hmdb"
Jul 17 17:23:50.058813 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:49.942140    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf00cc8_00f5_4664_8764_81e49ed6d9aa.slice/crio-ddd1850ed117d27398f66759daaeaa4c00c040dc566db37d069d04189f456f44 WatchSource:0}: Error finding container ddd1850ed117d27398f66759daaeaa4c00c040dc566db37d069d04189f456f44: Status 404 returned error can't find the container with id ddd1850ed117d27398f66759daaeaa4c00c040dc566db37d069d04189f456f44
Jul 17 17:23:50.058813 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:49.952704    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qgc79"
Jul 17 17:23:50.058813 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:23:49.962182    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode32df813_b6ae_45ff_9b71_82510269769b.slice/crio-561dcbd91905e324947ef36eca30c51366c80601d8d49dffa98fa0a99dbf0f1b WatchSource:0}: Error finding container 561dcbd91905e324947ef36eca30c51366c80601d8d49dffa98fa0a99dbf0f1b: Status 404 returned error can't find the container with id 561dcbd91905e324947ef36eca30c51366c80601d8d49dffa98fa0a99dbf0f1b
Jul 17 17:23:50.058813 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.055336    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:50.058813 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:50.055530    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:50.058813 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:50.055607    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs podName:8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d nodeName:}" failed. No retries permitted until 2026-07-17 17:23:51.055585743 +0000 UTC m=+3.060941595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs") pod "network-metrics-daemon-zvc9x" (UID: "8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:50.239541 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.156378    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:50.239541 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:50.156552    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:50.239541 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:50.156611    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret podName:7b5d7d3c-465d-40c3-8b32-d303103c7dd9 nodeName:}" failed. No retries permitted until 2026-07-17 17:23:51.156594979 +0000 UTC m=+3.161950819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret") pod "global-pull-secret-syncer-p8nmp" (UID: "7b5d7d3c-465d-40c3-8b32-d303103c7dd9") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:50.328299 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.257718    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:50.328299 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:50.257906    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:23:50.328299 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:50.257937    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:23:50.328299 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:50.257950    2572 projected.go:196] Error preparing data for projected volume kube-api-access-cvzj8 for pod openshift-network-diagnostics/network-check-target-2cr94: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:50.328299 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:50.258021    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8 podName:1b9381cd-6d3e-4100-bd3c-d159d489f3d9 nodeName:}" failed. No retries permitted until 2026-07-17 17:23:51.257997055 +0000 UTC m=+3.263352911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvzj8" (UniqueName: "kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8") pod "network-check-target-2cr94" (UID: "1b9381cd-6d3e-4100-bd3c-d159d489f3d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:50.461340 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.461231    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 17:18:50 +0000 UTC" deadline="2028-03-14 01:18:52.286109168 +0000 UTC"
Jul 17 17:23:50.461340 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.461291    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14527h55m1.824823165s"
Jul 17 17:23:50.513464 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.510118    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:50.513464 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:50.510281    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:23:50.550285 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.550214    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerStarted","Data":"3a9f7a9376bc37d3685749afa87e5abaecca34d5af776028df075eba47796857"}
Jul 17 17:23:50.557887 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.557536    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l2pbp" event={"ID":"12657b07-032c-4033-88ac-46e27583c859","Type":"ContainerStarted","Data":"019d316c9464d180c5f1be6bb9ba4368e8100cd487c6279f45cc963d2c38887e"}
Jul 17 17:23:50.564832 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.564726    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2xkgl" event={"ID":"bd33444c-515f-4f4d-a793-933561aee26d","Type":"ContainerStarted","Data":"3c0364e0a7f3d519291616c5aedbb6fa04d5321fa40e77f15ac73d6d02a58a49"}
Jul 17 17:23:50.578527 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.578344    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tf4tf" event={"ID":"4e09f9cb-3818-45a0-85ab-586d624a8935","Type":"ContainerStarted","Data":"284cb5df9a21f310b38e178a52a64dff03182f44889618a1433413c02cd024ab"}
Jul 17 17:23:50.586607 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.586482    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal" event={"ID":"6fa073bc5f1e1fe550b2b25522ade8d4","Type":"ContainerStarted","Data":"40ca81c6c7c9e374d2e538376feb8931ec5c390f6c65b05d6c9203b3a96f6623"}
Jul 17 17:23:50.595379 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.595331    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vbj8d" event={"ID":"ed5123f0-04f4-45e5-8098-6827fe05c426","Type":"ContainerStarted","Data":"7093f266e6672ce451523f06a154f13a2bc460da8124492743bd0f7c5519f9ef"}
Jul 17 17:23:50.603608 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.602055    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8hmdb" event={"ID":"aaf00cc8-00f5-4664-8764-81e49ed6d9aa","Type":"ContainerStarted","Data":"ddd1850ed117d27398f66759daaeaa4c00c040dc566db37d069d04189f456f44"}
Jul 17 17:23:50.607247 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.607108    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7" event={"ID":"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e","Type":"ContainerStarted","Data":"5270aefd2de6da8edeadaeaaddbf73e1cc03442d4d5a04824adf432117b069a9"}
Jul 17 17:23:50.609034 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.608995    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal" event={"ID":"73ecd14a99b2d4ac78d23a5262bad321","Type":"ContainerStarted","Data":"6f911bbc0490441174f9064b342ca71f1c5ca03a3fe25c3dfa49c18dcdc77c83"}
Jul 17 17:23:50.620111 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.620037    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgc79" event={"ID":"e32df813-b6ae-45ff-9b71-82510269769b","Type":"ContainerStarted","Data":"561dcbd91905e324947ef36eca30c51366c80601d8d49dffa98fa0a99dbf0f1b"}
Jul 17 17:23:50.631401 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.631309    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l9hsk" event={"ID":"1fb7e912-6649-48cc-88c7-989f172f937c","Type":"ContainerStarted","Data":"45431d5a231b5d71ef183a355715cb43c3f9b5d92e68ab3b98db021ea6e86274"}
Jul 17 17:23:50.819351 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:50.819310    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:23:51.066208 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:51.065409    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:51.066208 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.065634    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:51.066208 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.065708    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs podName:8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d nodeName:}" failed. No retries permitted until 2026-07-17 17:23:53.065687913 +0000 UTC m=+5.071043751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs") pod "network-metrics-daemon-zvc9x" (UID: "8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:51.166667 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:51.166476    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:51.166890 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.166686    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:51.166890 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.166775    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret podName:7b5d7d3c-465d-40c3-8b32-d303103c7dd9 nodeName:}" failed. No retries permitted until 2026-07-17 17:23:53.166752961 +0000 UTC m=+5.172108812 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret") pod "global-pull-secret-syncer-p8nmp" (UID: "7b5d7d3c-465d-40c3-8b32-d303103c7dd9") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:51.267249 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:51.267204    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:51.267646 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.267474    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:23:51.267646 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.267497    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:23:51.267646 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.267512    2572 projected.go:196] Error preparing data for projected volume kube-api-access-cvzj8 for pod openshift-network-diagnostics/network-check-target-2cr94: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:51.267646 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.267604    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8 podName:1b9381cd-6d3e-4100-bd3c-d159d489f3d9 nodeName:}" failed. No retries permitted until 2026-07-17 17:23:53.267583444 +0000 UTC m=+5.272939313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvzj8" (UniqueName: "kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8") pod "network-check-target-2cr94" (UID: "1b9381cd-6d3e-4100-bd3c-d159d489f3d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:51.462516 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:51.462411    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 17:18:50 +0000 UTC" deadline="2028-02-27 01:11:52.299087429 +0000 UTC"
Jul 17 17:23:51.462516 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:51.462460    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14143h48m0.83663263s"
Jul 17 17:23:51.510757 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:51.509872    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:51.510757 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.510050    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:23:51.510757 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:51.510506    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:51.510757 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:51.510628    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:23:52.519063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:52.519014    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:52.519561 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:52.519164    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:23:53.084716 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:53.084639    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:53.084917 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.084858    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:53.084987 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.084938    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs podName:8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d nodeName:}" failed. No retries permitted until 2026-07-17 17:23:57.084908815 +0000 UTC m=+9.090264653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs") pod "network-metrics-daemon-zvc9x" (UID: "8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:53.187048 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:53.186918    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:53.187248 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.187141    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:53.187248 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.187213    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret podName:7b5d7d3c-465d-40c3-8b32-d303103c7dd9 nodeName:}" failed. No retries permitted until 2026-07-17 17:23:57.187192355 +0000 UTC m=+9.192548196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret") pod "global-pull-secret-syncer-p8nmp" (UID: "7b5d7d3c-465d-40c3-8b32-d303103c7dd9") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:53.288055 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:53.287940    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:53.288248 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.288140    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:23:53.288248 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.288174    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:23:53.288248 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.288191    2572 projected.go:196] Error preparing data for projected volume kube-api-access-cvzj8 for pod openshift-network-diagnostics/network-check-target-2cr94: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:53.288428 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.288261    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8 podName:1b9381cd-6d3e-4100-bd3c-d159d489f3d9 nodeName:}" failed. No retries permitted until 2026-07-17 17:23:57.288239224 +0000 UTC m=+9.293595063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvzj8" (UniqueName: "kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8") pod "network-check-target-2cr94" (UID: "1b9381cd-6d3e-4100-bd3c-d159d489f3d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:53.510430 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:53.510340    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:53.510624 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.510488    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:23:53.510969 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:53.510943    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:53.511099 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:53.511059    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:23:54.513559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:54.513390    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:54.514171 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:54.513587    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:23:55.510175 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:55.510124    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:55.510373 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:55.510313    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:23:55.510965 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:55.510442    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:55.510965 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:55.510533    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:23:56.511360 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:56.510654    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:56.511360 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:56.510944    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:23:57.121821 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:57.121778    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:57.122034 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.121934    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:57.122034 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.122007    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs podName:8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d nodeName:}" failed. No retries permitted until 2026-07-17 17:24:05.121986414 +0000 UTC m=+17.127342254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs") pod "network-metrics-daemon-zvc9x" (UID: "8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:23:57.223484 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:57.222796    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:57.223484 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.223028    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:57.223484 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.223103    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret podName:7b5d7d3c-465d-40c3-8b32-d303103c7dd9 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:05.223079302 +0000 UTC m=+17.228435159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret") pod "global-pull-secret-syncer-p8nmp" (UID: "7b5d7d3c-465d-40c3-8b32-d303103c7dd9") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:23:57.323718 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:57.323616    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:57.323877 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.323798    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:23:57.323877 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.323822    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:23:57.323877 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.323838    2572 projected.go:196] Error preparing data for projected volume kube-api-access-cvzj8 for pod openshift-network-diagnostics/network-check-target-2cr94: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:57.324004 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.323907    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8 podName:1b9381cd-6d3e-4100-bd3c-d159d489f3d9 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:05.323888928 +0000 UTC m=+17.329244785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvzj8" (UniqueName: "kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8") pod "network-check-target-2cr94" (UID: "1b9381cd-6d3e-4100-bd3c-d159d489f3d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:23:57.509619 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:57.509576    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:57.509812 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:57.509591    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:57.509812 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.509732    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:23:57.509812 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:57.509777    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:23:58.511178 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:58.511140    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:23:58.511687 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:58.511368    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:23:59.509329 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:59.509288    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:23:59.509525 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:23:59.509417    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:23:59.509525 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:59.509427    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:23:59.509525 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:23:59.509507    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:00.509624 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:00.509587    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:00.510053 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:00.509725    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:01.509719 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:01.509673    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:01.510231 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:01.509673    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:01.510231 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:01.509808    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:01.510231 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:01.509867    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:02.510338 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:02.510286    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:02.510728 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:02.510425    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:03.509586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:03.509540    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:03.509780 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:03.509540    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:03.509780 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:03.509694    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:03.509904 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:03.509776    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:04.509899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:04.509856    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:04.510395 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:04.510051    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:05.188798 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:05.188749    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:05.188985 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.188888    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:24:05.188985 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.188971    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs podName:8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d nodeName:}" failed. No retries permitted until 2026-07-17 17:24:21.188952309 +0000 UTC m=+33.194308144 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs") pod "network-metrics-daemon-zvc9x" (UID: "8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:24:05.289594 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:05.289558    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:05.289828 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.289736    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:24:05.289828 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.289824    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret podName:7b5d7d3c-465d-40c3-8b32-d303103c7dd9 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:21.289802354 +0000 UTC m=+33.295158187 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret") pod "global-pull-secret-syncer-p8nmp" (UID: "7b5d7d3c-465d-40c3-8b32-d303103c7dd9") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:24:05.390811 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:05.390772    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:05.391011 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.390926    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:24:05.391011 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.390947    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:24:05.391011 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.390957    2572 projected.go:196] Error preparing data for projected volume kube-api-access-cvzj8 for pod openshift-network-diagnostics/network-check-target-2cr94: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:24:05.391147 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.391017    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8 podName:1b9381cd-6d3e-4100-bd3c-d159d489f3d9 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:21.390998915 +0000 UTC m=+33.396354767 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvzj8" (UniqueName: "kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8") pod "network-check-target-2cr94" (UID: "1b9381cd-6d3e-4100-bd3c-d159d489f3d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:24:05.509644 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:05.509560    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:05.509644 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:05.509605    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:05.509920 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.509681    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:05.509920 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:05.509793    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:06.510341 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:06.510298    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:06.510767 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:06.510424    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:07.509931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:07.509893    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:07.510110 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:07.510034    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:07.510171 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:07.510090    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:07.510240 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:07.510214    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:08.510287 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:08.510245    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:08.510801 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:08.510396    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:09.509761 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:09.509721    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:09.509938 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:09.509721    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:09.509938 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:09.509864    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:09.509938 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:09.509903    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:10.509656 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:10.509609    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:10.510134 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:10.509767    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:11.509850 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.509519    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:11.510525 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.509537    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:11.510525 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:11.509943    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:11.510525 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:11.510012    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:11.682411 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.682371    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal" event={"ID":"73ecd14a99b2d4ac78d23a5262bad321","Type":"ContainerStarted","Data":"03b1c1345362cf9b77f98ec2d8b6e2d710676785edb9d80843e5132e901f0186"}
Jul 17 17:24:11.684185 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.684158    2572 generic.go:356] "Generic (PLEG): container finished" podID="e32df813-b6ae-45ff-9b71-82510269769b" containerID="fcf038e1ab25d195340ecf8fb3146efe42d81dbf28542f3a885f3009750d42b5" exitCode=0
Jul 17 17:24:11.684426 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.684239    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgc79" event={"ID":"e32df813-b6ae-45ff-9b71-82510269769b","Type":"ContainerDied","Data":"fcf038e1ab25d195340ecf8fb3146efe42d81dbf28542f3a885f3009750d42b5"}
Jul 17 17:24:11.685874 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.685845    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l9hsk" event={"ID":"1fb7e912-6649-48cc-88c7-989f172f937c","Type":"ContainerStarted","Data":"651e19401737a37f9754c68dd785f123324f8170fc8f5cae9bb23750c5f3f61e"}
Jul 17 17:24:11.688876 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.688855    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:24:11.689246 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.689226    2572 generic.go:356] "Generic (PLEG): container finished" podID="00473330-68a9-4b11-8a60-732329b1891a" containerID="069f2a0e1bf757b2e8e099b8de8d14895b93f435262df0de56ddd05e5e269855" exitCode=1
Jul 17 17:24:11.689313 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.689299    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerStarted","Data":"c8b71bbb886fa40daf36347501fc88e3370d83b1d68b775b58ea2840fd126e1b"}
Jul 17 17:24:11.689377 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.689358    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerStarted","Data":"922608c5f95f48d22ba6b9dacf53167bc6a0ad6b4b3f6cac2e7f3e81445d51ce"}
Jul 17 17:24:11.689425 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.689413    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerStarted","Data":"00fa40d951c097d5e060581d2e770d0fba9ae08fe36b204c33d49dc704983b89"}
Jul 17 17:24:11.689462 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.689454    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerStarted","Data":"1377ec8fbed857047466b7c7cc8d829159dd19f993e55e7c8d3e764dbcad9e57"}
Jul 17 17:24:11.689514 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.689483    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerDied","Data":"069f2a0e1bf757b2e8e099b8de8d14895b93f435262df0de56ddd05e5e269855"}
Jul 17 17:24:11.689577 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.689536    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerStarted","Data":"5002f43ba2d635b3e5f6962bc3dfa28630da45e4151c91968d7efabad15c5621"}
Jul 17 17:24:11.691261 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.691235    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l2pbp" event={"ID":"12657b07-032c-4033-88ac-46e27583c859","Type":"ContainerStarted","Data":"67d3c4778e0e7ead3be02362dcf623745cc340629f3e8db270bd616df98ab0c5"}
Jul 17 17:24:11.692748 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.692726    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2xkgl" event={"ID":"bd33444c-515f-4f4d-a793-933561aee26d","Type":"ContainerStarted","Data":"f6d2248c4de7f55d29d8139287a829927a90c805c093d70ee98e9cdd8c32d638"}
Jul 17 17:24:11.694209 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.694183    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tf4tf" event={"ID":"4e09f9cb-3818-45a0-85ab-586d624a8935","Type":"ContainerStarted","Data":"d4c511b53d43caee4425fec28535353e2bc2c218597dbacfdf68316705549fb2"}
Jul 17 17:24:11.695939 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.695919    2572 generic.go:356] "Generic (PLEG): container finished" podID="6fa073bc5f1e1fe550b2b25522ade8d4" containerID="eeb5dd78d383a2595f3179f0825ae8a14c2fe21c10c0f1222137a72cb182de2c" exitCode=0
Jul 17 17:24:11.696031 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.695947    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal" event={"ID":"6fa073bc5f1e1fe550b2b25522ade8d4","Type":"ContainerDied","Data":"eeb5dd78d383a2595f3179f0825ae8a14c2fe21c10c0f1222137a72cb182de2c"}
Jul 17 17:24:11.697507 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.697476    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vbj8d" event={"ID":"ed5123f0-04f4-45e5-8098-6827fe05c426","Type":"ContainerStarted","Data":"742a33bcda487ea5c3161091c421ba1a73b75e998a8bc8fbfbca4bddf700141d"}
Jul 17 17:24:11.699060 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.699039    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7" event={"ID":"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e","Type":"ContainerStarted","Data":"2f3bbf871557b25c22850c7122bc1f51ec00ca93e43b220bcc4021cc363b8310"}
Jul 17 17:24:11.701634 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.701596    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-30.ec2.internal" podStartSLOduration=21.701584191 podStartE2EDuration="21.701584191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:11.70097542 +0000 UTC m=+23.706331275" watchObservedRunningTime="2026-07-17 17:24:11.701584191 +0000 UTC m=+23.706940046"
Jul 17 17:24:11.715710 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.715657    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-l9hsk" podStartSLOduration=10.030213865 podStartE2EDuration="22.715640181s" totalImagesPullingTime="12.685426316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:11.715615134 +0000 UTC m=+23.720970990" watchObservedRunningTime="2026-07-17 17:24:11.715640181 +0000 UTC m=+23.720996038"
Jul 17 17:24:11.753808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.753759    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l2pbp" podStartSLOduration=1.864790607 podStartE2EDuration="22.753741171s" totalImagesPullingTime="20.888950564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:11.753729139 +0000 UTC m=+23.759085006" watchObservedRunningTime="2026-07-17 17:24:11.753741171 +0000 UTC m=+23.759097026"
Jul 17 17:24:11.754274 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.754251    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vbj8d" podStartSLOduration=1.691778311 podStartE2EDuration="22.7542455s" totalImagesPullingTime="21.062467189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:11.731836866 +0000 UTC m=+23.737192721" watchObservedRunningTime="2026-07-17 17:24:11.7542455 +0000 UTC m=+23.759601352"
Jul 17 17:24:11.802149 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.802102    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tf4tf" podStartSLOduration=1.86113365 podStartE2EDuration="22.802084897s" totalImagesPullingTime="20.940951247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:11.801896275 +0000 UTC m=+23.807252129" watchObservedRunningTime="2026-07-17 17:24:11.802084897 +0000 UTC m=+23.807440752"
Jul 17 17:24:11.820099 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:11.820002    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2xkgl" podStartSLOduration=1.876074501 podStartE2EDuration="22.819985487s" totalImagesPullingTime="20.943910986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:11.819664277 +0000 UTC m=+23.825020129" watchObservedRunningTime="2026-07-17 17:24:11.819985487 +0000 UTC m=+23.825341341"
Jul 17 17:24:12.510032 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:12.509677    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:12.510475 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:12.510079    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:12.702687 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:12.702646    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8hmdb" event={"ID":"aaf00cc8-00f5-4664-8764-81e49ed6d9aa","Type":"ContainerStarted","Data":"9d6ca47c78692de6a15a08daf08ac5524fcba314fd20a34089c5d8efea94017a"}
Jul 17 17:24:12.732745 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:12.732685    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8hmdb" podStartSLOduration=2.922927724 podStartE2EDuration="23.732661648s" totalImagesPullingTime="20.809733924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:12.73198385 +0000 UTC m=+24.737339724" watchObservedRunningTime="2026-07-17 17:24:12.732661648 +0000 UTC m=+24.738017506"
Jul 17 17:24:12.879720 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:12.879684    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 17:24:13.499383 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:13.499251    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T17:24:12.879707788Z","UUID":"15e016f3-f761-43fd-acce-f2d6ca7c4f9c","Handler":null,"Name":"","Endpoint":""}
Jul 17 17:24:13.501165 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:13.501144    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 17:24:13.501325 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:13.501187    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 17:24:13.509460 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:13.509431    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:13.509460 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:13.509457    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:13.509698 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:13.509557    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:13.509698 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:13.509673    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:13.707031 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:13.706994    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7" event={"ID":"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e","Type":"ContainerStarted","Data":"ef930f50354d15b54ae422ca4729d42111c33648b6948de1c57f96c3487b74ba"}
Jul 17 17:24:13.709047 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:13.709017    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal" event={"ID":"6fa073bc5f1e1fe550b2b25522ade8d4","Type":"ContainerStarted","Data":"7e5ab62880f446de26672e85ed7a6fd366fb90c7d054be43b0405279c5ba27ca"}
Jul 17 17:24:13.724952 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:13.724893    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-30.ec2.internal" podStartSLOduration=23.72487075 podStartE2EDuration="23.72487075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:13.72466586 +0000 UTC m=+25.730021717" watchObservedRunningTime="2026-07-17 17:24:13.72487075 +0000 UTC m=+25.730226608"
Jul 17 17:24:14.146549 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:14.146519    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:24:14.147278 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:14.147253    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:24:14.510421 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:14.510327    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:14.510657 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:14.510481    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:14.715407 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:14.715372    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:24:14.715957 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:14.715848    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerStarted","Data":"de2fa0307111f1a8d5a2154602092e7ec8cdb52dd69dc26b950c7ab60ed14025"}
Jul 17 17:24:14.718019 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:14.717991    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7" event={"ID":"363a912a-bf5d-4ab7-9c2b-21698b1c4f1e","Type":"ContainerStarted","Data":"ed3db6fe3654a9e57236c8bfbd625c1fb852640f9e17bc9ab00c098da46e4456"}
Jul 17 17:24:14.718559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:14.718440    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:24:14.718696 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:14.718677    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vbj8d"
Jul 17 17:24:14.735847 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:14.735790    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fnb7" podStartSLOduration=2.2385391 podStartE2EDuration="25.735772106s" totalImagesPullingTime="23.497233006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:14.735110873 +0000 UTC m=+26.740466731" watchObservedRunningTime="2026-07-17 17:24:14.735772106 +0000 UTC m=+26.741128030"
Jul 17 17:24:15.509998 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:15.509956    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:15.509998 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:15.510000    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:15.510258 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:15.510116    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:15.510332 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:15.510266    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:16.510134 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:16.510088    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:16.510674 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:16.510255    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:17.509472 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.509281    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:17.509644 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.509319    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:17.509644 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:17.509599    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:17.509644 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:17.509629    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:17.731203 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.731124    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:24:17.732116 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.731527    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerStarted","Data":"3038ad74f47e1e5bfa5c026fd9b15c644d71bcf993df4756e7c188e1eeaa030d"}
Jul 17 17:24:17.732116 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.731833    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:24:17.732116 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.731902    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:24:17.732116 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.732019    2572 scope.go:117] "RemoveContainer" containerID="069f2a0e1bf757b2e8e099b8de8d14895b93f435262df0de56ddd05e5e269855"
Jul 17 17:24:17.733601 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.733574    2572 generic.go:356] "Generic (PLEG): container finished" podID="e32df813-b6ae-45ff-9b71-82510269769b" containerID="18dfce2f2a2bad633e1fcb3c4fc9cbe1509d4a8a7339a59ddb135a5a50d4b992" exitCode=0
Jul 17 17:24:17.733899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.733613    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgc79" event={"ID":"e32df813-b6ae-45ff-9b71-82510269769b","Type":"ContainerDied","Data":"18dfce2f2a2bad633e1fcb3c4fc9cbe1509d4a8a7339a59ddb135a5a50d4b992"}
Jul 17 17:24:17.749729 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:17.749708    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:24:18.510427 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:18.510213    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:18.510766 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:18.510532    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:18.739411 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:18.739378    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:24:18.739837 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:18.739733    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" event={"ID":"00473330-68a9-4b11-8a60-732329b1891a","Type":"ContainerStarted","Data":"0f30b172b5284fc9c7cd0d4289245f70b46dc778fbd8cdcc9c95e8312a3646bd"}
Jul 17 17:24:18.740114 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:18.740091    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:24:18.761559 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:18.761463    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:24:18.772942 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:18.772890    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm" podStartSLOduration=8.776777239 podStartE2EDuration="29.772873062s" totalImagesPullingTime="20.996095823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:18.772370502 +0000 UTC m=+30.777726362" watchObservedRunningTime="2026-07-17 17:24:18.772873062 +0000 UTC m=+30.778228916"
Jul 17 17:24:19.338395 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:19.338355    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zvc9x"]
Jul 17 17:24:19.338586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:19.338506    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:19.338655 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:19.338610    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:19.342787 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:19.342757    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p8nmp"]
Jul 17 17:24:19.342898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:19.342878    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:19.343010 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:19.342987    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:19.343422 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:19.343399    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2cr94"]
Jul 17 17:24:19.343529 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:19.343515    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:19.343622 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:19.343605    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:19.744518 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:19.744402    2572 generic.go:356] "Generic (PLEG): container finished" podID="e32df813-b6ae-45ff-9b71-82510269769b" containerID="586d2b359135c2b5306419b79d4d51655c2c9c888dbff9b2425410c9015182c0" exitCode=0
Jul 17 17:24:19.744518 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:19.744503    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgc79" event={"ID":"e32df813-b6ae-45ff-9b71-82510269769b","Type":"ContainerDied","Data":"586d2b359135c2b5306419b79d4d51655c2c9c888dbff9b2425410c9015182c0"}
Jul 17 17:24:21.214990 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:21.214947    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:21.215462 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.215124    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:24:21.215462 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.215209    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs podName:8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d nodeName:}" failed. No retries permitted until 2026-07-17 17:24:53.215187221 +0000 UTC m=+65.220543060 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs") pod "network-metrics-daemon-zvc9x" (UID: "8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:24:21.315614 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:21.315566    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:21.315790 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.315714    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:24:21.315790 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.315779    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret podName:7b5d7d3c-465d-40c3-8b32-d303103c7dd9 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:53.315763212 +0000 UTC m=+65.321119044 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret") pod "global-pull-secret-syncer-p8nmp" (UID: "7b5d7d3c-465d-40c3-8b32-d303103c7dd9") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:24:21.416788 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:21.416746    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:21.416970 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.416889    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:24:21.416970 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.416908    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:24:21.416970 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.416918    2572 projected.go:196] Error preparing data for projected volume kube-api-access-cvzj8 for pod openshift-network-diagnostics/network-check-target-2cr94: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:24:21.417073 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.416976    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8 podName:1b9381cd-6d3e-4100-bd3c-d159d489f3d9 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:53.416962188 +0000 UTC m=+65.422318021 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvzj8" (UniqueName: "kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8") pod "network-check-target-2cr94" (UID: "1b9381cd-6d3e-4100-bd3c-d159d489f3d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:24:21.509814 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:21.509725    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:21.509814 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:21.509765    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:21.510031 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:21.509832    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:21.510031 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.509837    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:21.510031 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.509901    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:21.510031 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:21.509966    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:21.751201 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:21.751168    2572 generic.go:356] "Generic (PLEG): container finished" podID="e32df813-b6ae-45ff-9b71-82510269769b" containerID="a9a8d07668aab9cb83f7cec442e114f5c9727eddb7dbc9250d69e796d895db07" exitCode=0
Jul 17 17:24:21.751372 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:21.751228    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgc79" event={"ID":"e32df813-b6ae-45ff-9b71-82510269769b","Type":"ContainerDied","Data":"a9a8d07668aab9cb83f7cec442e114f5c9727eddb7dbc9250d69e796d895db07"}
Jul 17 17:24:23.510057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.510020    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:23.510057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.510056    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:23.510600 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:23.510159    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cr94" podUID="1b9381cd-6d3e-4100-bd3c-d159d489f3d9"
Jul 17 17:24:23.510600 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.510211    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:23.510600 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:23.510354    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvc9x" podUID="8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d"
Jul 17 17:24:23.510600 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:23.510431    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8nmp" podUID="7b5d7d3c-465d-40c3-8b32-d303103c7dd9"
Jul 17 17:24:23.814130 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.814099    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-30.ec2.internal" event="NodeReady"
Jul 17 17:24:23.814328 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.814258    2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 17:24:23.848300 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.848260    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85d4d7bb84-vg4h2"]
Jul 17 17:24:23.852058 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.852035    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:23.854819 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.854787    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 17 17:24:23.855155 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.855134    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 17 17:24:23.855605 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.855582    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kbx6g\""
Jul 17 17:24:23.855730 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.855714    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 17 17:24:23.862284 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.861881    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-74tnw"]
Jul 17 17:24:23.863667 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.863635    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 17 17:24:23.866299 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.866269    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85d4d7bb84-vg4h2"]
Jul 17 17:24:23.866451 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.866429    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:23.866702 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.866677    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-djgz7"]
Jul 17 17:24:23.870628 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.870602    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 17:24:23.870778 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.870687    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-smb8x\""
Jul 17 17:24:23.870778 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.870715    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-djgz7"
Jul 17 17:24:23.870896 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.870808    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 17:24:23.873320 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.873296    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 17:24:23.873472 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.873354    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 17:24:23.873472 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.873404    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 17:24:23.873669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.873628    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mcwhs\""
Jul 17 17:24:23.876517 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.876480    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-74tnw"]
Jul 17 17:24:23.878333 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.878308    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-djgz7"]
Jul 17 17:24:23.935035 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.934990    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-bound-sa-token\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:23.935035 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935030    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfsk5\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-kube-api-access-mfsk5\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:23.935268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935109    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-image-registry-private-configuration\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:23.935268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935138    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-trusted-ca\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:23.935268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935164    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de66d357-8e8a-4315-9db4-9adf5d86d420-config-volume\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:23.935268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935192    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-installation-pull-secrets\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:23.935268 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935229    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:23.935467 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935286    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7wlp\" (UniqueName: \"kubernetes.io/projected/f15bcab1-4a49-45b7-94a7-f7175f8ce2c5-kube-api-access-q7wlp\") pod \"ingress-canary-djgz7\" (UID: \"f15bcab1-4a49-45b7-94a7-f7175f8ce2c5\") " pod="openshift-ingress-canary/ingress-canary-djgz7"
Jul 17 17:24:23.935467 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935302    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:23.935467 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935323    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de66d357-8e8a-4315-9db4-9adf5d86d420-tmp-dir\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:23.935467 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935373    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f15bcab1-4a49-45b7-94a7-f7175f8ce2c5-cert\") pod \"ingress-canary-djgz7\" (UID: \"f15bcab1-4a49-45b7-94a7-f7175f8ce2c5\") " pod="openshift-ingress-canary/ingress-canary-djgz7"
Jul 17 17:24:23.935467 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935392    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3e18948-e8c7-4229-9f8b-7322ed643b9f-ca-trust-extracted\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:23.935467 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935419    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-certificates\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:23.935467 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.935441    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2knk\" (UniqueName: \"kubernetes.io/projected/de66d357-8e8a-4315-9db4-9adf5d86d420-kube-api-access-c2knk\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:23.988916 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:23.988875    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vbj8d_ed5123f0-04f4-45e5-8098-6827fe05c426/konnectivity-agent/0.log" limit=1024
Jul 17 17:24:24.035964 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.035929    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-bound-sa-token\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.035964 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.035966    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfsk5\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-kube-api-access-mfsk5\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.036205 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036007    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-image-registry-private-configuration\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.036205 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036032    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-trusted-ca\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.036205 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036054    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de66d357-8e8a-4315-9db4-9adf5d86d420-config-volume\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:24.036205 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036079    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-installation-pull-secrets\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.036205 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036105    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.036205 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036131    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7wlp\" (UniqueName: \"kubernetes.io/projected/f15bcab1-4a49-45b7-94a7-f7175f8ce2c5-kube-api-access-q7wlp\") pod \"ingress-canary-djgz7\" (UID: \"f15bcab1-4a49-45b7-94a7-f7175f8ce2c5\") " pod="openshift-ingress-canary/ingress-canary-djgz7"
Jul 17 17:24:24.036205 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036151    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:24.036205 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036187    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de66d357-8e8a-4315-9db4-9adf5d86d420-tmp-dir\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:24.036598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036234    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f15bcab1-4a49-45b7-94a7-f7175f8ce2c5-cert\") pod \"ingress-canary-djgz7\" (UID: \"f15bcab1-4a49-45b7-94a7-f7175f8ce2c5\") " pod="openshift-ingress-canary/ingress-canary-djgz7"
Jul 17 17:24:24.036598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036262    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3e18948-e8c7-4229-9f8b-7322ed643b9f-ca-trust-extracted\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.036598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036287    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-certificates\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.036598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036313    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2knk\" (UniqueName: \"kubernetes.io/projected/de66d357-8e8a-4315-9db4-9adf5d86d420-kube-api-access-c2knk\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:24.037069 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036724    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de66d357-8e8a-4315-9db4-9adf5d86d420-tmp-dir\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:24.037069 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.036912    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de66d357-8e8a-4315-9db4-9adf5d86d420-config-volume\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:24.037069 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.036930    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 17:24:24.037069 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.037045    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d4d7bb84-vg4h2: secret "image-registry-tls" not found
Jul 17 17:24:24.037354 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.037096    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls podName:f3e18948-e8c7-4229-9f8b-7322ed643b9f nodeName:}" failed. No retries permitted until 2026-07-17 17:24:24.537078642 +0000 UTC m=+36.542434478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls") pod "image-registry-85d4d7bb84-vg4h2" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f") : secret "image-registry-tls" not found
Jul 17 17:24:24.037354 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.036989    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 17:24:24.037354 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.037179    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls podName:de66d357-8e8a-4315-9db4-9adf5d86d420 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:24.537122282 +0000 UTC m=+36.542478115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls") pod "dns-default-74tnw" (UID: "de66d357-8e8a-4315-9db4-9adf5d86d420") : secret "dns-default-metrics-tls" not found
Jul 17 17:24:24.037547 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.037386    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3e18948-e8c7-4229-9f8b-7322ed643b9f-ca-trust-extracted\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.037935 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.037903    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-certificates\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.038370 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.038345    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-trusted-ca\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.041482 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.041454    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-installation-pull-secrets\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.041482 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.041467    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-image-registry-private-configuration\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.041658 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.041548    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f15bcab1-4a49-45b7-94a7-f7175f8ce2c5-cert\") pod \"ingress-canary-djgz7\" (UID: \"f15bcab1-4a49-45b7-94a7-f7175f8ce2c5\") " pod="openshift-ingress-canary/ingress-canary-djgz7"
Jul 17 17:24:24.046795 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.046762    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2knk\" (UniqueName: \"kubernetes.io/projected/de66d357-8e8a-4315-9db4-9adf5d86d420-kube-api-access-c2knk\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:24.047534 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.047474    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-bound-sa-token\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.048507 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.048465    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7wlp\" (UniqueName: \"kubernetes.io/projected/f15bcab1-4a49-45b7-94a7-f7175f8ce2c5-kube-api-access-q7wlp\") pod \"ingress-canary-djgz7\" (UID: \"f15bcab1-4a49-45b7-94a7-f7175f8ce2c5\") " pod="openshift-ingress-canary/ingress-canary-djgz7"
Jul 17 17:24:24.049204 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.049185    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfsk5\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-kube-api-access-mfsk5\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.074021 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.073939    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vbj8d_ed5123f0-04f4-45e5-8098-6827fe05c426/konnectivity-agent/0.log" limit=1024
Jul 17 17:24:24.187219 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.187186    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-djgz7"
Jul 17 17:24:24.352582 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.352476    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-djgz7"]
Jul 17 17:24:24.358130 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:24:24.358087    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15bcab1_4a49_45b7_94a7_f7175f8ce2c5.slice/crio-dd35e3fc4df4ce3ad72f1f7556eb03c4d734ba33a1f5603005c2be5cc66990f2 WatchSource:0}: Error finding container dd35e3fc4df4ce3ad72f1f7556eb03c4d734ba33a1f5603005c2be5cc66990f2: Status 404 returned error can't find the container with id dd35e3fc4df4ce3ad72f1f7556eb03c4d734ba33a1f5603005c2be5cc66990f2
Jul 17 17:24:24.540333 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.540280    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:24.541203 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.540337    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:24.541203 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.540415    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 17:24:24.541203 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.540439    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d4d7bb84-vg4h2: secret "image-registry-tls" not found
Jul 17 17:24:24.541203 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.540537    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls podName:f3e18948-e8c7-4229-9f8b-7322ed643b9f nodeName:}" failed. No retries permitted until 2026-07-17 17:24:25.540516579 +0000 UTC m=+37.545872417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls") pod "image-registry-85d4d7bb84-vg4h2" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f") : secret "image-registry-tls" not found
Jul 17 17:24:24.541203 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.540547    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 17:24:24.541203 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:24.540603    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls podName:de66d357-8e8a-4315-9db4-9adf5d86d420 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:25.540586352 +0000 UTC m=+37.545942190 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls") pod "dns-default-74tnw" (UID: "de66d357-8e8a-4315-9db4-9adf5d86d420") : secret "dns-default-metrics-tls" not found
Jul 17 17:24:24.760118 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:24.760029    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-djgz7" event={"ID":"f15bcab1-4a49-45b7-94a7-f7175f8ce2c5","Type":"ContainerStarted","Data":"dd35e3fc4df4ce3ad72f1f7556eb03c4d734ba33a1f5603005c2be5cc66990f2"}
Jul 17 17:24:25.509830 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.509733    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:25.509830 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.509767    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:25.509830 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.509785    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:25.513076 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.513045    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 17:24:25.513227 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.513165    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 17:24:25.514625 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.514602    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nlczc\""
Jul 17 17:24:25.514840 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.514821    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 17:24:25.515447 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.515019    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 17:24:25.515447 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.515205    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-srf5x\""
Jul 17 17:24:25.549280 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.549244    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:25.549747 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:25.549398    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:25.549747 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:25.549416    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 17:24:25.549747 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:25.549545    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 17:24:25.549747 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:25.549560    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d4d7bb84-vg4h2: secret "image-registry-tls" not found
Jul 17 17:24:25.549747 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:25.549569    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls podName:de66d357-8e8a-4315-9db4-9adf5d86d420 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:27.549546855 +0000 UTC m=+39.554902693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls") pod "dns-default-74tnw" (UID: "de66d357-8e8a-4315-9db4-9adf5d86d420") : secret "dns-default-metrics-tls" not found
Jul 17 17:24:25.549747 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:25.549611    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls podName:f3e18948-e8c7-4229-9f8b-7322ed643b9f nodeName:}" failed. No retries permitted until 2026-07-17 17:24:27.54959518 +0000 UTC m=+39.554951031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls") pod "image-registry-85d4d7bb84-vg4h2" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f") : secret "image-registry-tls" not found
Jul 17 17:24:27.565757 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:27.565720    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:27.565757 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:27.565760    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:27.566212 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:27.565875    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 17:24:27.566212 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:27.565902    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d4d7bb84-vg4h2: secret "image-registry-tls" not found
Jul 17 17:24:27.566212 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:27.565886    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 17:24:27.566212 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:27.565957    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls podName:f3e18948-e8c7-4229-9f8b-7322ed643b9f nodeName:}" failed. No retries permitted until 2026-07-17 17:24:31.565940134 +0000 UTC m=+43.571295972 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls") pod "image-registry-85d4d7bb84-vg4h2" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f") : secret "image-registry-tls" not found
Jul 17 17:24:27.566212 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:27.566018    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls podName:de66d357-8e8a-4315-9db4-9adf5d86d420 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:31.566006291 +0000 UTC m=+43.571362123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls") pod "dns-default-74tnw" (UID: "de66d357-8e8a-4315-9db4-9adf5d86d420") : secret "dns-default-metrics-tls" not found
Jul 17 17:24:27.909578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:27.909267    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vbj8d_ed5123f0-04f4-45e5-8098-6827fe05c426/konnectivity-agent/0.log" limit=1024
Jul 17 17:24:28.009646 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:28.009615    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vbj8d_ed5123f0-04f4-45e5-8098-6827fe05c426/konnectivity-agent/0.log" limit=1024
Jul 17 17:24:28.771243 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:28.771203    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-djgz7" event={"ID":"f15bcab1-4a49-45b7-94a7-f7175f8ce2c5","Type":"ContainerStarted","Data":"1196b7ccec2d7d2aa8a1083e8fe25b7d1814379a59fe70905ef4e1d30e6ae692"}
Jul 17 17:24:28.773886 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:28.773856    2572 generic.go:356] "Generic (PLEG): container finished" podID="e32df813-b6ae-45ff-9b71-82510269769b" containerID="3972047ef2ca2d68e0072ab3efe35aaf5945436502d0d1022ce89405c8f352c6" exitCode=0
Jul 17 17:24:28.774034 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:28.773915    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgc79" event={"ID":"e32df813-b6ae-45ff-9b71-82510269769b","Type":"ContainerDied","Data":"3972047ef2ca2d68e0072ab3efe35aaf5945436502d0d1022ce89405c8f352c6"}
Jul 17 17:24:28.787875 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:28.787827    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-djgz7" podStartSLOduration=2.378788154 podStartE2EDuration="5.787810756s" totalImagesPullingTime="3.409022602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:24:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:28.786699481 +0000 UTC m=+40.792055336" watchObservedRunningTime="2026-07-17 17:24:28.787810756 +0000 UTC m=+40.793166610"
Jul 17 17:24:29.779392 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:29.779353    2572 generic.go:356] "Generic (PLEG): container finished" podID="e32df813-b6ae-45ff-9b71-82510269769b" containerID="0c36f6b1c69b1727334f0b4132d64358bf972329d6bc524ccf4be3e1020b379b" exitCode=0
Jul 17 17:24:29.779892 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:29.779447    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgc79" event={"ID":"e32df813-b6ae-45ff-9b71-82510269769b","Type":"ContainerDied","Data":"0c36f6b1c69b1727334f0b4132d64358bf972329d6bc524ccf4be3e1020b379b"}
Jul 17 17:24:30.785119 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:30.785079    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qgc79" event={"ID":"e32df813-b6ae-45ff-9b71-82510269769b","Type":"ContainerStarted","Data":"277c068af0b3384797424b4a796467a51f2342f3a93edcce3ea1b0b465a5715e"}
Jul 17 17:24:30.809057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:30.808994    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qgc79" podStartSLOduration=7.286146989 podStartE2EDuration="41.808979392s" totalImagesPullingTime="34.522832403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:30.807819493 +0000 UTC m=+42.813175358" watchObservedRunningTime="2026-07-17 17:24:30.808979392 +0000 UTC m=+42.814335247"
Jul 17 17:24:30.931287 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:30.931230    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vbj8d_ed5123f0-04f4-45e5-8098-6827fe05c426/konnectivity-agent/0.log" limit=1024
Jul 17 17:24:30.991266 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:30.991227    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vbj8d_ed5123f0-04f4-45e5-8098-6827fe05c426/konnectivity-agent/0.log" limit=1024
Jul 17 17:24:31.595049 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:31.595002    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:31.595049 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:31.595056    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:31.595277 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:31.595179    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 17:24:31.595277 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:31.595193    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d4d7bb84-vg4h2: secret "image-registry-tls" not found
Jul 17 17:24:31.595277 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:31.595219    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 17:24:31.595277 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:31.595250    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls podName:f3e18948-e8c7-4229-9f8b-7322ed643b9f nodeName:}" failed. No retries permitted until 2026-07-17 17:24:39.595233184 +0000 UTC m=+51.600589020 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls") pod "image-registry-85d4d7bb84-vg4h2" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f") : secret "image-registry-tls" not found
Jul 17 17:24:31.595277 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:31.595272    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls podName:de66d357-8e8a-4315-9db4-9adf5d86d420 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:39.595259228 +0000 UTC m=+51.600615065 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls") pod "dns-default-74tnw" (UID: "de66d357-8e8a-4315-9db4-9adf5d86d420") : secret "dns-default-metrics-tls" not found
Jul 17 17:24:33.962393 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:33.962357    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vbj8d_ed5123f0-04f4-45e5-8098-6827fe05c426/konnectivity-agent/0.log" limit=1024
Jul 17 17:24:39.654734 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:39.654664    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:39.654734 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:39.654735    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:39.655245 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:39.654829    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 17:24:39.655245 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:39.654849    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d4d7bb84-vg4h2: secret "image-registry-tls" not found
Jul 17 17:24:39.655245 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:39.654852    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 17:24:39.655245 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:39.654904    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls podName:de66d357-8e8a-4315-9db4-9adf5d86d420 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:55.654889824 +0000 UTC m=+67.660245658 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls") pod "dns-default-74tnw" (UID: "de66d357-8e8a-4315-9db4-9adf5d86d420") : secret "dns-default-metrics-tls" not found
Jul 17 17:24:39.655245 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:39.654918    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls podName:f3e18948-e8c7-4229-9f8b-7322ed643b9f nodeName:}" failed. No retries permitted until 2026-07-17 17:24:55.65491166 +0000 UTC m=+67.660267492 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls") pod "image-registry-85d4d7bb84-vg4h2" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f") : secret "image-registry-tls" not found
Jul 17 17:24:50.757656 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:50.757629    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbmxm"
Jul 17 17:24:53.256218 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.256157    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:24:53.258840 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.258820    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 17:24:53.267077 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:53.267045    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 17:24:53.267154 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:53.267137    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs podName:8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d nodeName:}" failed. No retries permitted until 2026-07-17 17:25:57.267119115 +0000 UTC m=+129.272474947 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs") pod "network-metrics-daemon-zvc9x" (UID: "8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d") : secret "metrics-daemon-secret" not found
Jul 17 17:24:53.357255 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.357210    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:53.360311 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.360292    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 17:24:53.370276 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.370248    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b5d7d3c-465d-40c3-8b32-d303103c7dd9-original-pull-secret\") pod \"global-pull-secret-syncer-p8nmp\" (UID: \"7b5d7d3c-465d-40c3-8b32-d303103c7dd9\") " pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:53.440000 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.439959    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8nmp"
Jul 17 17:24:53.458146 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.458111    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:53.461170 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.461146    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 17:24:53.471506 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.471455    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 17:24:53.482066 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.482037    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvzj8\" (UniqueName: \"kubernetes.io/projected/1b9381cd-6d3e-4100-bd3c-d159d489f3d9-kube-api-access-cvzj8\") pod \"network-check-target-2cr94\" (UID: \"1b9381cd-6d3e-4100-bd3c-d159d489f3d9\") " pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:53.577760 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.577728    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p8nmp"]
Jul 17 17:24:53.582519 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:24:53.581983    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b5d7d3c_465d_40c3_8b32_d303103c7dd9.slice/crio-97101c5305edde237d383dc11fb7be910864f25f8a5be375bbc815cdd5020e60 WatchSource:0}: Error finding container 97101c5305edde237d383dc11fb7be910864f25f8a5be375bbc815cdd5020e60: Status 404 returned error can't find the container with id 97101c5305edde237d383dc11fb7be910864f25f8a5be375bbc815cdd5020e60
Jul 17 17:24:53.735277 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.735249    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-srf5x\""
Jul 17 17:24:53.743378 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.743352    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:24:53.840986 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.840944    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p8nmp" event={"ID":"7b5d7d3c-465d-40c3-8b32-d303103c7dd9","Type":"ContainerStarted","Data":"97101c5305edde237d383dc11fb7be910864f25f8a5be375bbc815cdd5020e60"}
Jul 17 17:24:53.868465 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.868433    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2cr94"]
Jul 17 17:24:53.871454 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:24:53.871425    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9381cd_6d3e_4100_bd3c_d159d489f3d9.slice/crio-38cf215283a042b5667612f9b2da03e02def77b6fa2fb6d51fbf4a38110a8369 WatchSource:0}: Error finding container 38cf215283a042b5667612f9b2da03e02def77b6fa2fb6d51fbf4a38110a8369: Status 404 returned error can't find the container with id 38cf215283a042b5667612f9b2da03e02def77b6fa2fb6d51fbf4a38110a8369
Jul 17 17:24:53.923416 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.923382    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4"]
Jul 17 17:24:53.925650 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.925633    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-kgqwx"]
Jul 17 17:24:53.925774 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.925756    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4"
Jul 17 17:24:53.928379 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.928359    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:53.928520 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.928437    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-prlk8\""
Jul 17 17:24:53.928788 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.928767    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 17 17:24:53.928866 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.928830    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 17 17:24:53.930735 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.930715    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 17 17:24:53.930863 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.930847    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 17 17:24:53.930924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.930897    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wf84g\""
Jul 17 17:24:53.931909 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.931890    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 17:24:53.932102 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.932086    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 17:24:53.938191 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.938090    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4"]
Jul 17 17:24:53.939013 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.938993    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 17 17:24:53.939098 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:53.939032    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-kgqwx"]
Jul 17 17:24:54.018977 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.018935    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252"]
Jul 17 17:24:54.020993 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.020968    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252"
Jul 17 17:24:54.024432 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.024405    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-n2q8j\""
Jul 17 17:24:54.024632 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.024522    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"]
Jul 17 17:24:54.026505 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.026471    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-p75lc"]
Jul 17 17:24:54.026612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.026582    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.028098 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.028077    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.028996 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.028976    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 17 17:24:54.029373 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.029357    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 17:24:54.029427 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.029404    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 17:24:54.029427 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.029418    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6gnzn\""
Jul 17 17:24:54.029533 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.029439    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 17 17:24:54.030863 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.030844    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 17 17:24:54.031151 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.031139    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 17 17:24:54.031476 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.031460    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vd5dk\""
Jul 17 17:24:54.031841 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.031822    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 17 17:24:54.032571 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.032554    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 17 17:24:54.036895 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.036875    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 17 17:24:54.040896 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.040833    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252"]
Jul 17 17:24:54.043457 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.043436    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"]
Jul 17 17:24:54.044633 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.044608    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-p75lc"]
Jul 17 17:24:54.064258 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.064223    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3189c-2328-4a13-85f6-0cb7423020cc-service-ca-bundle\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.064431 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.064264    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skln7\" (UniqueName: \"kubernetes.io/projected/bcf3189c-2328-4a13-85f6-0cb7423020cc-kube-api-access-skln7\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.064431 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.064303    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr7lm\" (UniqueName: \"kubernetes.io/projected/f5dd4a69-b165-44db-91e8-3317b4f37dd0-kube-api-access-cr7lm\") pod \"volume-data-source-validator-78cf4c6f6-9fxx4\" (UID: \"f5dd4a69-b165-44db-91e8-3317b4f37dd0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4"
Jul 17 17:24:54.064431 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.064338    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3189c-2328-4a13-85f6-0cb7423020cc-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.064431 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.064356    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf3189c-2328-4a13-85f6-0cb7423020cc-serving-cert\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.064582 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.064437    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcf3189c-2328-4a13-85f6-0cb7423020cc-tmp\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.064582 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.064471    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bcf3189c-2328-4a13-85f6-0cb7423020cc-snapshots\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.120386 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.120353    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"]
Jul 17 17:24:54.122202 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.122186    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:24:54.124677 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.124646    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qhsls\""
Jul 17 17:24:54.124923 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.124908    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 17 17:24:54.125101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.125089    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 17 17:24:54.134641 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.134614    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"]
Jul 17 17:24:54.165294 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165252    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3189c-2328-4a13-85f6-0cb7423020cc-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.165294 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/26681e78-0726-442e-8390-d6c9917d77da-config\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.165598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165348    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-467qx\" (UniqueName: \"kubernetes.io/projected/26681e78-0726-442e-8390-d6c9917d77da-kube-api-access-467qx\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.165598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165382    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bcf3189c-2328-4a13-85f6-0cb7423020cc-snapshots\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.165598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165421    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr7lm\" (UniqueName: \"kubernetes.io/projected/f5dd4a69-b165-44db-91e8-3317b4f37dd0-kube-api-access-cr7lm\") pod \"volume-data-source-validator-78cf4c6f6-9fxx4\" (UID: \"f5dd4a69-b165-44db-91e8-3317b4f37dd0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4"
Jul 17 17:24:54.165598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165452    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/642842c7-dd0b-4a6a-a358-eeea433a7d25-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.165598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165506    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26681e78-0726-442e-8390-d6c9917d77da-serving-cert\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.165598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165580    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf3189c-2328-4a13-85f6-0cb7423020cc-serving-cert\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.165898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165618    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skln7\" (UniqueName: \"kubernetes.io/projected/bcf3189c-2328-4a13-85f6-0cb7423020cc-kube-api-access-skln7\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.165898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165668    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcf3189c-2328-4a13-85f6-0cb7423020cc-tmp\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.165898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165717    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.165898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165754    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3189c-2328-4a13-85f6-0cb7423020cc-service-ca-bundle\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.165898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165782    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w6qm\" (UniqueName: \"kubernetes.io/projected/642842c7-dd0b-4a6a-a358-eeea433a7d25-kube-api-access-9w6qm\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.165898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165845    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k79r2\" (UniqueName: \"kubernetes.io/projected/5970a9a4-567b-4794-831d-b66b30b791ab-kube-api-access-k79r2\") pod \"network-check-source-7bc5b6bf85-kf252\" (UID: \"5970a9a4-567b-4794-831d-b66b30b791ab\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252"
Jul 17 17:24:54.165898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.165880    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26681e78-0726-442e-8390-d6c9917d77da-trusted-ca\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.166258 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.166193    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcf3189c-2328-4a13-85f6-0cb7423020cc-tmp\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.166292 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.166251    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3189c-2328-4a13-85f6-0cb7423020cc-service-ca-bundle\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.166333 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.166289    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bcf3189c-2328-4a13-85f6-0cb7423020cc-snapshots\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.166441 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.166422    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3189c-2328-4a13-85f6-0cb7423020cc-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.168413 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.168390    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf3189c-2328-4a13-85f6-0cb7423020cc-serving-cert\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.174989 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.174952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skln7\" (UniqueName: \"kubernetes.io/projected/bcf3189c-2328-4a13-85f6-0cb7423020cc-kube-api-access-skln7\") pod \"insights-operator-555d9d7885-kgqwx\" (UID: \"bcf3189c-2328-4a13-85f6-0cb7423020cc\") " pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.175119 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.175006    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr7lm\" (UniqueName: \"kubernetes.io/projected/f5dd4a69-b165-44db-91e8-3317b4f37dd0-kube-api-access-cr7lm\") pod \"volume-data-source-validator-78cf4c6f6-9fxx4\" (UID: \"f5dd4a69-b165-44db-91e8-3317b4f37dd0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4"
Jul 17 17:24:54.237986 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.237947    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4"
Jul 17 17:24:54.244422 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.244386    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-kgqwx"
Jul 17 17:24:54.266750 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.266715    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.266764    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/642842c7-dd0b-4a6a-a358-eeea433a7d25-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.266804    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26681e78-0726-442e-8390-d6c9917d77da-serving-cert\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.266877    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.266910    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w6qm\" (UniqueName: \"kubernetes.io/projected/642842c7-dd0b-4a6a-a358-eeea433a7d25-kube-api-access-9w6qm\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.266946    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k79r2\" (UniqueName: \"kubernetes.io/projected/5970a9a4-567b-4794-831d-b66b30b791ab-kube-api-access-k79r2\") pod \"network-check-source-7bc5b6bf85-kf252\" (UID: \"5970a9a4-567b-4794-831d-b66b30b791ab\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.266978    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26681e78-0726-442e-8390-d6c9917d77da-trusted-ca\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.267015    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/26681e78-0726-442e-8390-d6c9917d77da-config\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:54.267036    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.267051    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/638d220b-0afb-46b6-8826-4110aeb4940b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.267082    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-467qx\" (UniqueName: \"kubernetes.io/projected/26681e78-0726-442e-8390-d6c9917d77da-kube-api-access-467qx\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.267184 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:54.267122    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls podName:642842c7-dd0b-4a6a-a358-eeea433a7d25 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:54.767100138 +0000 UTC m=+66.772455972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-p8rl8" (UID: "642842c7-dd0b-4a6a-a358-eeea433a7d25") : secret "cluster-monitoring-operator-tls" not found
Jul 17 17:24:54.268286 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.268158    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/26681e78-0726-442e-8390-d6c9917d77da-config\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.268286 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.268246    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/642842c7-dd0b-4a6a-a358-eeea433a7d25-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.268574 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.268343    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26681e78-0726-442e-8390-d6c9917d77da-trusted-ca\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.269509 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.269475    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26681e78-0726-442e-8390-d6c9917d77da-serving-cert\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.277045 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.277016    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-467qx\" (UniqueName: \"kubernetes.io/projected/26681e78-0726-442e-8390-d6c9917d77da-kube-api-access-467qx\") pod \"console-operator-565b7b569d-p75lc\" (UID: \"26681e78-0726-442e-8390-d6c9917d77da\") " pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.277224 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.277203    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w6qm\" (UniqueName: \"kubernetes.io/projected/642842c7-dd0b-4a6a-a358-eeea433a7d25-kube-api-access-9w6qm\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.278757 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.278731    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k79r2\" (UniqueName: \"kubernetes.io/projected/5970a9a4-567b-4794-831d-b66b30b791ab-kube-api-access-k79r2\") pod \"network-check-source-7bc5b6bf85-kf252\" (UID: \"5970a9a4-567b-4794-831d-b66b30b791ab\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252"
Jul 17 17:24:54.330618 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.330563    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252"
Jul 17 17:24:54.346718 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.346661    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:24:54.376946 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.376897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:24:54.377137 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:54.377045    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 17:24:54.377137 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.377082    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/638d220b-0afb-46b6-8826-4110aeb4940b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:24:54.377258 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:54.377160    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert podName:638d220b-0afb-46b6-8826-4110aeb4940b nodeName:}" failed. No retries permitted until 2026-07-17 17:24:54.877136757 +0000 UTC m=+66.882492597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-98j2x" (UID: "638d220b-0afb-46b6-8826-4110aeb4940b") : secret "networking-console-plugin-cert" not found
Jul 17 17:24:54.378046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.377997    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/638d220b-0afb-46b6-8826-4110aeb4940b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:24:54.411115 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.411078    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4"]
Jul 17 17:24:54.416959 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:24:54.416919    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5dd4a69_b165_44db_91e8_3317b4f37dd0.slice/crio-caf6d18f95a6adbb1346b4e3b49cd6061ba943d3d09b1b486ce0fa534bc336a3 WatchSource:0}: Error finding container caf6d18f95a6adbb1346b4e3b49cd6061ba943d3d09b1b486ce0fa534bc336a3: Status 404 returned error can't find the container with id caf6d18f95a6adbb1346b4e3b49cd6061ba943d3d09b1b486ce0fa534bc336a3
Jul 17 17:24:54.423215 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.423132    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-kgqwx"]
Jul 17 17:24:54.501454 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.501420    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252"]
Jul 17 17:24:54.506397 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:24:54.506363    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5970a9a4_567b_4794_831d_b66b30b791ab.slice/crio-6c11aa6ad369df9b7f642b96afcfdb488a445c42d2948d76d42f469e68297587 WatchSource:0}: Error finding container 6c11aa6ad369df9b7f642b96afcfdb488a445c42d2948d76d42f469e68297587: Status 404 returned error can't find the container with id 6c11aa6ad369df9b7f642b96afcfdb488a445c42d2948d76d42f469e68297587
Jul 17 17:24:54.528249 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.525967    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-p75lc"]
Jul 17 17:24:54.531107 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:24:54.531069    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26681e78_0726_442e_8390_d6c9917d77da.slice/crio-2ff08d5cdc43ae9b6ae02727f15627103765a9df126586d021ef9934bf95f340 WatchSource:0}: Error finding container 2ff08d5cdc43ae9b6ae02727f15627103765a9df126586d021ef9934bf95f340: Status 404 returned error can't find the container with id 2ff08d5cdc43ae9b6ae02727f15627103765a9df126586d021ef9934bf95f340
Jul 17 17:24:54.781150 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.781110    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:54.781330 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:54.781293    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 17:24:54.781400 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:54.781375    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls podName:642842c7-dd0b-4a6a-a358-eeea433a7d25 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:55.781356347 +0000 UTC m=+67.786712195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-p8rl8" (UID: "642842c7-dd0b-4a6a-a358-eeea433a7d25") : secret "cluster-monitoring-operator-tls" not found
Jul 17 17:24:54.845854 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.845773    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252" event={"ID":"5970a9a4-567b-4794-831d-b66b30b791ab","Type":"ContainerStarted","Data":"6c11aa6ad369df9b7f642b96afcfdb488a445c42d2948d76d42f469e68297587"}
Jul 17 17:24:54.847476 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.847433    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4" event={"ID":"f5dd4a69-b165-44db-91e8-3317b4f37dd0","Type":"ContainerStarted","Data":"caf6d18f95a6adbb1346b4e3b49cd6061ba943d3d09b1b486ce0fa534bc336a3"}
Jul 17 17:24:54.848885 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.848849    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2cr94" event={"ID":"1b9381cd-6d3e-4100-bd3c-d159d489f3d9","Type":"ContainerStarted","Data":"38cf215283a042b5667612f9b2da03e02def77b6fa2fb6d51fbf4a38110a8369"}
Jul 17 17:24:54.850286 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.850257    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-kgqwx" event={"ID":"bcf3189c-2328-4a13-85f6-0cb7423020cc","Type":"ContainerStarted","Data":"bbb5a5d8b7f8a6fec9c824021f8a40d4b4cc3c1d9fe3c10a6d4002e0366e8c98"}
Jul 17 17:24:54.851655 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.851621    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-p75lc" event={"ID":"26681e78-0726-442e-8390-d6c9917d77da","Type":"ContainerStarted","Data":"2ff08d5cdc43ae9b6ae02727f15627103765a9df126586d021ef9934bf95f340"}
Jul 17 17:24:54.882448 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:54.882395    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:24:54.882695 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:54.882675    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 17:24:54.882839 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:54.882753    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert podName:638d220b-0afb-46b6-8826-4110aeb4940b nodeName:}" failed. No retries permitted until 2026-07-17 17:24:55.882732842 +0000 UTC m=+67.888088681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-98j2x" (UID: "638d220b-0afb-46b6-8826-4110aeb4940b") : secret "networking-console-plugin-cert" not found
Jul 17 17:24:55.691160 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:55.690267    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:24:55.691160 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:55.690390    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:24:55.691160 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:55.690571    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 17:24:55.691160 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:55.690636    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls podName:de66d357-8e8a-4315-9db4-9adf5d86d420 nodeName:}" failed. No retries permitted until 2026-07-17 17:25:27.690616147 +0000 UTC m=+99.695971983 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls") pod "dns-default-74tnw" (UID: "de66d357-8e8a-4315-9db4-9adf5d86d420") : secret "dns-default-metrics-tls" not found
Jul 17 17:24:55.691160 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:55.691056    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 17:24:55.691160 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:55.691074    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d4d7bb84-vg4h2: secret "image-registry-tls" not found
Jul 17 17:24:55.691160 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:55.691123    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls podName:f3e18948-e8c7-4229-9f8b-7322ed643b9f nodeName:}" failed. No retries permitted until 2026-07-17 17:25:27.691106381 +0000 UTC m=+99.696462217 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls") pod "image-registry-85d4d7bb84-vg4h2" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f") : secret "image-registry-tls" not found
Jul 17 17:24:55.791240 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:55.791198    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:55.791445 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:55.791383    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 17:24:55.791539 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:55.791451    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls podName:642842c7-dd0b-4a6a-a358-eeea433a7d25 nodeName:}" failed. No retries permitted until 2026-07-17 17:24:57.791430477 +0000 UTC m=+69.796786315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-p8rl8" (UID: "642842c7-dd0b-4a6a-a358-eeea433a7d25") : secret "cluster-monitoring-operator-tls" not found
Jul 17 17:24:55.892207 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:55.892170    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:24:55.892465 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:55.892428    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 17:24:55.892593 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:55.892526    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert podName:638d220b-0afb-46b6-8826-4110aeb4940b nodeName:}" failed. No retries permitted until 2026-07-17 17:24:57.892503804 +0000 UTC m=+69.897859651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-98j2x" (UID: "638d220b-0afb-46b6-8826-4110aeb4940b") : secret "networking-console-plugin-cert" not found
Jul 17 17:24:56.861295 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:56.861226    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4" event={"ID":"f5dd4a69-b165-44db-91e8-3317b4f37dd0","Type":"ContainerStarted","Data":"e46b4dd23303a189c852ea831cb2a2920addb4bbee1a767aab54f71c9f832055"}
Jul 17 17:24:56.878716 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:56.878628    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9fxx4" podStartSLOduration=2.108770769 podStartE2EDuration="3.878605634s" totalImagesPullingTime="1.769834865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:24:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:24:56.875954451 +0000 UTC m=+68.881310331" watchObservedRunningTime="2026-07-17 17:24:56.878605634 +0000 UTC m=+68.883961491"
Jul 17 17:24:57.812115 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:57.812077    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:24:57.812302 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:57.812200    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 17:24:57.812302 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:57.812255    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls podName:642842c7-dd0b-4a6a-a358-eeea433a7d25 nodeName:}" failed. No retries permitted until 2026-07-17 17:25:01.812240561 +0000 UTC m=+73.817596393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-p8rl8" (UID: "642842c7-dd0b-4a6a-a358-eeea433a7d25") : secret "cluster-monitoring-operator-tls" not found
Jul 17 17:24:57.912507 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:24:57.912463    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:24:57.912892 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:57.912624    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 17:24:57.912892 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:24:57.912697    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert podName:638d220b-0afb-46b6-8826-4110aeb4940b nodeName:}" failed. No retries permitted until 2026-07-17 17:25:01.912678481 +0000 UTC m=+73.918034314 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-98j2x" (UID: "638d220b-0afb-46b6-8826-4110aeb4940b") : secret "networking-console-plugin-cert" not found
Jul 17 17:25:00.634033 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.633938    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-fvtf9"]
Jul 17 17:25:00.636076 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.636057    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.638792 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.638764    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 17 17:25:00.640057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.639978    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 17 17:25:00.640057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.639978    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-g7v4b\""
Jul 17 17:25:00.640057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.640036    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 17 17:25:00.640290 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.639981    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 17 17:25:00.646194 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.646168    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-fvtf9"]
Jul 17 17:25:00.737228 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.737183    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ec60effa-c697-4589-b29c-0193f9b4776f-signing-cabundle\") pod \"service-ca-58598695f7-fvtf9\" (UID: \"ec60effa-c697-4589-b29c-0193f9b4776f\") " pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.737437 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.737275    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfbb5\" (UniqueName: \"kubernetes.io/projected/ec60effa-c697-4589-b29c-0193f9b4776f-kube-api-access-qfbb5\") pod \"service-ca-58598695f7-fvtf9\" (UID: \"ec60effa-c697-4589-b29c-0193f9b4776f\") " pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.737437 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.737328    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ec60effa-c697-4589-b29c-0193f9b4776f-signing-key\") pod \"service-ca-58598695f7-fvtf9\" (UID: \"ec60effa-c697-4589-b29c-0193f9b4776f\") " pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.837759 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.837716    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ec60effa-c697-4589-b29c-0193f9b4776f-signing-key\") pod \"service-ca-58598695f7-fvtf9\" (UID: \"ec60effa-c697-4589-b29c-0193f9b4776f\") " pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.837956 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.837866    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ec60effa-c697-4589-b29c-0193f9b4776f-signing-cabundle\") pod \"service-ca-58598695f7-fvtf9\" (UID: \"ec60effa-c697-4589-b29c-0193f9b4776f\") " pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.837956 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.837912    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfbb5\" (UniqueName: \"kubernetes.io/projected/ec60effa-c697-4589-b29c-0193f9b4776f-kube-api-access-qfbb5\") pod \"service-ca-58598695f7-fvtf9\" (UID: \"ec60effa-c697-4589-b29c-0193f9b4776f\") " pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.838686 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.838662    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ec60effa-c697-4589-b29c-0193f9b4776f-signing-cabundle\") pod \"service-ca-58598695f7-fvtf9\" (UID: \"ec60effa-c697-4589-b29c-0193f9b4776f\") " pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.840289 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.840263    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ec60effa-c697-4589-b29c-0193f9b4776f-signing-key\") pod \"service-ca-58598695f7-fvtf9\" (UID: \"ec60effa-c697-4589-b29c-0193f9b4776f\") " pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.845737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.845713    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfbb5\" (UniqueName: \"kubernetes.io/projected/ec60effa-c697-4589-b29c-0193f9b4776f-kube-api-access-qfbb5\") pod \"service-ca-58598695f7-fvtf9\" (UID: \"ec60effa-c697-4589-b29c-0193f9b4776f\") " pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.876191 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.876159    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-kgqwx" event={"ID":"bcf3189c-2328-4a13-85f6-0cb7423020cc","Type":"ContainerStarted","Data":"a18735a4b72f1b8559a6f7b02a1ffa21f732954458ca528f81858b58f78759ce"}
Jul 17 17:25:00.877939 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.877918    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/0.log"
Jul 17 17:25:00.878059 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.877955    2572 generic.go:356] "Generic (PLEG): container finished" podID="26681e78-0726-442e-8390-d6c9917d77da" containerID="cf024f84149d37cca690a73a3764b3bdd7c8a1432fecc9e350820818f67d13e0" exitCode=255
Jul 17 17:25:00.878059 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.878016    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-p75lc" event={"ID":"26681e78-0726-442e-8390-d6c9917d77da","Type":"ContainerDied","Data":"cf024f84149d37cca690a73a3764b3bdd7c8a1432fecc9e350820818f67d13e0"}
Jul 17 17:25:00.878287 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.878265    2572 scope.go:117] "RemoveContainer" containerID="cf024f84149d37cca690a73a3764b3bdd7c8a1432fecc9e350820818f67d13e0"
Jul 17 17:25:00.879651 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.879624    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252" event={"ID":"5970a9a4-567b-4794-831d-b66b30b791ab","Type":"ContainerStarted","Data":"34e031f579d3940102274de20c0c7d3cff757eb64522ad88837245d5882f57e8"}
Jul 17 17:25:00.881238 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.881210    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p8nmp" event={"ID":"7b5d7d3c-465d-40c3-8b32-d303103c7dd9","Type":"ContainerStarted","Data":"53befed95c08e182d4f4b3740a14533e78dae63695468f7bcc73fb37b8bda9f6"}
Jul 17 17:25:00.882921 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.882897    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2cr94" event={"ID":"1b9381cd-6d3e-4100-bd3c-d159d489f3d9","Type":"ContainerStarted","Data":"3552d1cfda05544bf123920519dbe9118cb1e6ede8d14e9c0086dbb61b5d8478"}
Jul 17 17:25:00.883055 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.883035    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:25:00.895269 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.895222    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-kgqwx" podStartSLOduration=1.943140288 podStartE2EDuration="7.895179813s" totalImagesPullingTime="5.952039525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:24:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:00.894994038 +0000 UTC m=+72.900349905" watchObservedRunningTime="2026-07-17 17:25:00.895179813 +0000 UTC m=+72.900535669"
Jul 17 17:25:00.911902 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.911716    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kf252" podStartSLOduration=1.022524899 podStartE2EDuration="6.91169537s" totalImagesPullingTime="5.889170471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:00.911392319 +0000 UTC m=+72.916748175" watchObservedRunningTime="2026-07-17 17:25:00.91169537 +0000 UTC m=+72.917051229"
Jul 17 17:25:00.946412 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.946384    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-fvtf9"
Jul 17 17:25:00.951623 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.951570    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p8nmp" podStartSLOduration=65.144664156 podStartE2EDuration="1m11.9515488s" totalImagesPullingTime="6.806884644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:00.949757834 +0000 UTC m=+72.955113685" watchObservedRunningTime="2026-07-17 17:25:00.9515488 +0000 UTC m=+72.956904657"
Jul 17 17:25:00.972175 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:00.972113    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2cr94" podStartSLOduration=65.457667743 podStartE2EDuration="1m11.972090872s" totalImagesPullingTime="6.514423129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:00.970836403 +0000 UTC m=+72.976192268" watchObservedRunningTime="2026-07-17 17:25:00.972090872 +0000 UTC m=+72.977446728"
Jul 17 17:25:01.096961 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.096923    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-fvtf9"]
Jul 17 17:25:01.100122 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:25:01.100092    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec60effa_c697_4589_b29c_0193f9b4776f.slice/crio-3652a162f29c70d3acf836d4e66535f0bda7596c475ffa44d4085993d396b632 WatchSource:0}: Error finding container 3652a162f29c70d3acf836d4e66535f0bda7596c475ffa44d4085993d396b632: Status 404 returned error can't find the container with id 3652a162f29c70d3acf836d4e66535f0bda7596c475ffa44d4085993d396b632
Jul 17 17:25:01.847338 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.847296    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:25:01.849533 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:01.847889    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 17:25:01.849533 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:01.847986    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls podName:642842c7-dd0b-4a6a-a358-eeea433a7d25 nodeName:}" failed. No retries permitted until 2026-07-17 17:25:09.847963873 +0000 UTC m=+81.853319724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-p8rl8" (UID: "642842c7-dd0b-4a6a-a358-eeea433a7d25") : secret "cluster-monitoring-operator-tls" not found
Jul 17 17:25:01.887758 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.887716    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-fvtf9" event={"ID":"ec60effa-c697-4589-b29c-0193f9b4776f","Type":"ContainerStarted","Data":"3652a162f29c70d3acf836d4e66535f0bda7596c475ffa44d4085993d396b632"}
Jul 17 17:25:01.889330 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.889307    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:25:01.889737 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.889718    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/0.log"
Jul 17 17:25:01.889852 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.889761    2572 generic.go:356] "Generic (PLEG): container finished" podID="26681e78-0726-442e-8390-d6c9917d77da" containerID="34c9bc31a7b5fbfe9461fcfe57202e0d96df078c3d85f3004f7a42c0d89856f5" exitCode=255
Jul 17 17:25:01.889904 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.889860    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-p75lc" event={"ID":"26681e78-0726-442e-8390-d6c9917d77da","Type":"ContainerDied","Data":"34c9bc31a7b5fbfe9461fcfe57202e0d96df078c3d85f3004f7a42c0d89856f5"}
Jul 17 17:25:01.889951 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.889922    2572 scope.go:117] "RemoveContainer" containerID="cf024f84149d37cca690a73a3764b3bdd7c8a1432fecc9e350820818f67d13e0"
Jul 17 17:25:01.890093 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.890075    2572 scope.go:117] "RemoveContainer" containerID="34c9bc31a7b5fbfe9461fcfe57202e0d96df078c3d85f3004f7a42c0d89856f5"
Jul 17 17:25:01.890306 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:01.890284    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-p75lc_openshift-console-operator(26681e78-0726-442e-8390-d6c9917d77da)\"" pod="openshift-console-operator/console-operator-565b7b569d-p75lc" podUID="26681e78-0726-442e-8390-d6c9917d77da"
Jul 17 17:25:01.948177 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:01.948124    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:25:01.948348 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:01.948294    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 17:25:01.948402 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:01.948389    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert podName:638d220b-0afb-46b6-8826-4110aeb4940b nodeName:}" failed. No retries permitted until 2026-07-17 17:25:09.948361387 +0000 UTC m=+81.953717241 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-98j2x" (UID: "638d220b-0afb-46b6-8826-4110aeb4940b") : secret "networking-console-plugin-cert" not found
Jul 17 17:25:02.896644 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:02.896611    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:25:02.897093 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:02.897073    2572 scope.go:117] "RemoveContainer" containerID="34c9bc31a7b5fbfe9461fcfe57202e0d96df078c3d85f3004f7a42c0d89856f5"
Jul 17 17:25:02.897313 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:02.897285    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-p75lc_openshift-console-operator(26681e78-0726-442e-8390-d6c9917d77da)\"" pod="openshift-console-operator/console-operator-565b7b569d-p75lc" podUID="26681e78-0726-442e-8390-d6c9917d77da"
Jul 17 17:25:03.529630 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:03.529607    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l2pbp_12657b07-032c-4033-88ac-46e27583c859/dns-node-resolver/0.log"
Jul 17 17:25:03.901997 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:03.901959    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-fvtf9" event={"ID":"ec60effa-c697-4589-b29c-0193f9b4776f","Type":"ContainerStarted","Data":"1636b6cca3cee2de43bd5cf7b14506fcf1bf4d05fb5960a82996e4097f613bbc"}
Jul 17 17:25:03.918448 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:03.918397    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-fvtf9" podStartSLOduration=1.955773513 podStartE2EDuration="3.918380045s" totalImagesPullingTime="1.962606532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:03.917195912 +0000 UTC m=+75.922551786" watchObservedRunningTime="2026-07-17 17:25:03.918380045 +0000 UTC m=+75.923735900"
Jul 17 17:25:04.347197 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:04.347151    2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:25:04.347412 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:04.347235    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:25:04.347718 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:04.347701    2572 scope.go:117] "RemoveContainer" containerID="34c9bc31a7b5fbfe9461fcfe57202e0d96df078c3d85f3004f7a42c0d89856f5"
Jul 17 17:25:04.347913 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:04.347890    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-p75lc_openshift-console-operator(26681e78-0726-442e-8390-d6c9917d77da)\"" pod="openshift-console-operator/console-operator-565b7b569d-p75lc" podUID="26681e78-0726-442e-8390-d6c9917d77da"
Jul 17 17:25:04.538744 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:04.538715    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l9hsk_1fb7e912-6649-48cc-88c7-989f172f937c/node-ca/0.log"
Jul 17 17:25:05.138815 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:05.138779    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-djgz7_f15bcab1-4a49-45b7-94a7-f7175f8ce2c5/serve-healthcheck-canary/0.log"
Jul 17 17:25:09.918045 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:09.917984    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:25:09.918602 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:09.918177    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 17:25:09.918602 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:09.918272    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls podName:642842c7-dd0b-4a6a-a358-eeea433a7d25 nodeName:}" failed. No retries permitted until 2026-07-17 17:25:25.91824938 +0000 UTC m=+97.923605217 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-p8rl8" (UID: "642842c7-dd0b-4a6a-a358-eeea433a7d25") : secret "cluster-monitoring-operator-tls" not found
Jul 17 17:25:10.018608 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:10.018566    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:25:10.018826 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:10.018750    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 17:25:10.018899 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:25:10.018836    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert podName:638d220b-0afb-46b6-8826-4110aeb4940b nodeName:}" failed. No retries permitted until 2026-07-17 17:25:26.018812766 +0000 UTC m=+98.024168605 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-98j2x" (UID: "638d220b-0afb-46b6-8826-4110aeb4940b") : secret "networking-console-plugin-cert" not found
Jul 17 17:25:17.509979 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:17.509821    2572 scope.go:117] "RemoveContainer" containerID="34c9bc31a7b5fbfe9461fcfe57202e0d96df078c3d85f3004f7a42c0d89856f5"
Jul 17 17:25:17.953962 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:17.953936    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:25:17.954149 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:17.954007    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-p75lc" event={"ID":"26681e78-0726-442e-8390-d6c9917d77da","Type":"ContainerStarted","Data":"9ea3dc717e719557c062575ae2bbbfac3c3f9d6e1d3aaf4a0d463bb1aa32eab7"}
Jul 17 17:25:17.954304 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:17.954282    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:25:17.973933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:17.973878    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-p75lc" podStartSLOduration=18.126712805 podStartE2EDuration="23.97386265s" totalImagesPullingTime="5.847149845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:17.973725955 +0000 UTC m=+89.979081812" watchObservedRunningTime="2026-07-17 17:25:17.97386265 +0000 UTC m=+89.979218504"
Jul 17 17:25:18.080235 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:18.080206    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-p75lc"
Jul 17 17:25:22.445157 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.445093    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"]
Jul 17 17:25:22.448088 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.448063    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"]
Jul 17 17:25:22.448247 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.448184    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"
Jul 17 17:25:22.450712 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.450689    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-tpkj6\""
Jul 17 17:25:22.451025 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.451008    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.452278 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.452257    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 17 17:25:22.452405 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.452267    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 17 17:25:22.452405 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.452284    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 17 17:25:22.452405 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.452389    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 17 17:25:22.453538 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.453516    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 17 17:25:22.460288 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.460261    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"]
Jul 17 17:25:22.461354 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.461332    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"]
Jul 17 17:25:22.519861 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.519825    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3eccb27e-8b40-4ed3-a893-537bba16e278-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc6d9484b-dbqhv\" (UID: \"3eccb27e-8b40-4ed3-a893-537bba16e278\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.520022 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.519934    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5qhc\" (UniqueName: \"kubernetes.io/projected/966fa4a8-a03e-41d1-a09e-177b9ae6542f-kube-api-access-l5qhc\") pod \"managed-serviceaccount-addon-agent-66bcbc75fd-qqnst\" (UID: \"966fa4a8-a03e-41d1-a09e-177b9ae6542f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"
Jul 17 17:25:22.520022 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.519995    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqxz6\" (UniqueName: \"kubernetes.io/projected/3eccb27e-8b40-4ed3-a893-537bba16e278-kube-api-access-fqxz6\") pod \"klusterlet-addon-workmgr-5fc6d9484b-dbqhv\" (UID: \"3eccb27e-8b40-4ed3-a893-537bba16e278\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.520094 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.520042    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/966fa4a8-a03e-41d1-a09e-177b9ae6542f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66bcbc75fd-qqnst\" (UID: \"966fa4a8-a03e-41d1-a09e-177b9ae6542f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"
Jul 17 17:25:22.520094 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.520073    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3eccb27e-8b40-4ed3-a893-537bba16e278-tmp\") pod \"klusterlet-addon-workmgr-5fc6d9484b-dbqhv\" (UID: \"3eccb27e-8b40-4ed3-a893-537bba16e278\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.529953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.529917    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d9xdg"]
Jul 17 17:25:22.533594 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.533572    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.537161 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.537139    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 17:25:22.537399 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.537382    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j69cs\""
Jul 17 17:25:22.537946 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.537930    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 17:25:22.584236 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.584198    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-jmwvg"]
Jul 17 17:25:22.587289 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.587270    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-jmwvg"
Jul 17 17:25:22.596567 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.596534    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 17:25:22.599882 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.599857    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gswcv\""
Jul 17 17:25:22.599989 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.599904    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 17:25:22.613111 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.613081    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-jmwvg"]
Jul 17 17:25:22.620603 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620572    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3eccb27e-8b40-4ed3-a893-537bba16e278-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc6d9484b-dbqhv\" (UID: \"3eccb27e-8b40-4ed3-a893-537bba16e278\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.620754 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620614    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf7s7\" (UniqueName: \"kubernetes.io/projected/3c455fa4-d71d-4748-abe2-6c903f6ae0f3-kube-api-access-gf7s7\") pod \"downloads-5bc67dcd46-jmwvg\" (UID: \"3c455fa4-d71d-4748-abe2-6c903f6ae0f3\") " pod="openshift-console/downloads-5bc67dcd46-jmwvg"
Jul 17 17:25:22.620754 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620643    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e98b193a-4711-4e94-b692-d0d36fe172d3-crio-socket\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.620754 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620705    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5qhc\" (UniqueName: \"kubernetes.io/projected/966fa4a8-a03e-41d1-a09e-177b9ae6542f-kube-api-access-l5qhc\") pod \"managed-serviceaccount-addon-agent-66bcbc75fd-qqnst\" (UID: \"966fa4a8-a03e-41d1-a09e-177b9ae6542f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"
Jul 17 17:25:22.620865 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620762    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98b193a-4711-4e94-b692-d0d36fe172d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.620865 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620811    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzn5h\" (UniqueName: \"kubernetes.io/projected/e98b193a-4711-4e94-b692-d0d36fe172d3-kube-api-access-pzn5h\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.620865 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620838    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98b193a-4711-4e94-b692-d0d36fe172d3-data-volume\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.620970 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620866    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98b193a-4711-4e94-b692-d0d36fe172d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.620970 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620888    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqxz6\" (UniqueName: \"kubernetes.io/projected/3eccb27e-8b40-4ed3-a893-537bba16e278-kube-api-access-fqxz6\") pod \"klusterlet-addon-workmgr-5fc6d9484b-dbqhv\" (UID: \"3eccb27e-8b40-4ed3-a893-537bba16e278\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.620970 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620930    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/966fa4a8-a03e-41d1-a09e-177b9ae6542f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66bcbc75fd-qqnst\" (UID: \"966fa4a8-a03e-41d1-a09e-177b9ae6542f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"
Jul 17 17:25:22.620970 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.620960    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3eccb27e-8b40-4ed3-a893-537bba16e278-tmp\") pod \"klusterlet-addon-workmgr-5fc6d9484b-dbqhv\" (UID: \"3eccb27e-8b40-4ed3-a893-537bba16e278\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.621348 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.621331    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3eccb27e-8b40-4ed3-a893-537bba16e278-tmp\") pod \"klusterlet-addon-workmgr-5fc6d9484b-dbqhv\" (UID: \"3eccb27e-8b40-4ed3-a893-537bba16e278\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.623165 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.623143    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3eccb27e-8b40-4ed3-a893-537bba16e278-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc6d9484b-dbqhv\" (UID: \"3eccb27e-8b40-4ed3-a893-537bba16e278\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.623276 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.623267    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/966fa4a8-a03e-41d1-a09e-177b9ae6542f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66bcbc75fd-qqnst\" (UID: \"966fa4a8-a03e-41d1-a09e-177b9ae6542f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"
Jul 17 17:25:22.636324 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.636277    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d9xdg"]
Jul 17 17:25:22.651250 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.651213    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqxz6\" (UniqueName: \"kubernetes.io/projected/3eccb27e-8b40-4ed3-a893-537bba16e278-kube-api-access-fqxz6\") pod \"klusterlet-addon-workmgr-5fc6d9484b-dbqhv\" (UID: \"3eccb27e-8b40-4ed3-a893-537bba16e278\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.654314 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.654284    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5qhc\" (UniqueName: \"kubernetes.io/projected/966fa4a8-a03e-41d1-a09e-177b9ae6542f-kube-api-access-l5qhc\") pod \"managed-serviceaccount-addon-agent-66bcbc75fd-qqnst\" (UID: \"966fa4a8-a03e-41d1-a09e-177b9ae6542f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"
Jul 17 17:25:22.721818 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.721690    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98b193a-4711-4e94-b692-d0d36fe172d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.721818 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.721770    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzn5h\" (UniqueName: \"kubernetes.io/projected/e98b193a-4711-4e94-b692-d0d36fe172d3-kube-api-access-pzn5h\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.721818 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.721799    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98b193a-4711-4e94-b692-d0d36fe172d3-data-volume\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.722104 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.721834    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98b193a-4711-4e94-b692-d0d36fe172d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.722104 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.721929    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf7s7\" (UniqueName: \"kubernetes.io/projected/3c455fa4-d71d-4748-abe2-6c903f6ae0f3-kube-api-access-gf7s7\") pod \"downloads-5bc67dcd46-jmwvg\" (UID: \"3c455fa4-d71d-4748-abe2-6c903f6ae0f3\") " pod="openshift-console/downloads-5bc67dcd46-jmwvg"
Jul 17 17:25:22.722104 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.721953    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e98b193a-4711-4e94-b692-d0d36fe172d3-crio-socket\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.722480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.722386    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e98b193a-4711-4e94-b692-d0d36fe172d3-crio-socket\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.722658 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.722627    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98b193a-4711-4e94-b692-d0d36fe172d3-data-volume\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.727364 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.727339    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98b193a-4711-4e94-b692-d0d36fe172d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.727481 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.727448    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98b193a-4711-4e94-b692-d0d36fe172d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.731405 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.731377    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzn5h\" (UniqueName: \"kubernetes.io/projected/e98b193a-4711-4e94-b692-d0d36fe172d3-kube-api-access-pzn5h\") pod \"insights-runtime-extractor-d9xdg\" (UID: \"e98b193a-4711-4e94-b692-d0d36fe172d3\") " pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.732746 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.732717    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf7s7\" (UniqueName: \"kubernetes.io/projected/3c455fa4-d71d-4748-abe2-6c903f6ae0f3-kube-api-access-gf7s7\") pod \"downloads-5bc67dcd46-jmwvg\" (UID: \"3c455fa4-d71d-4748-abe2-6c903f6ae0f3\") " pod="openshift-console/downloads-5bc67dcd46-jmwvg"
Jul 17 17:25:22.769999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.769966    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"
Jul 17 17:25:22.775359 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.775341    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:22.843055 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.842974    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d9xdg"
Jul 17 17:25:22.895912 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.895837    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-jmwvg"
Jul 17 17:25:22.935934 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.935870    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst"]
Jul 17 17:25:22.940303 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:25:22.940237    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod966fa4a8_a03e_41d1_a09e_177b9ae6542f.slice/crio-a8d16b8677eeb39f922c2f577dfe84b140c4f840affd036c1a5feb1df20d21b6 WatchSource:0}: Error finding container a8d16b8677eeb39f922c2f577dfe84b140c4f840affd036c1a5feb1df20d21b6: Status 404 returned error can't find the container with id a8d16b8677eeb39f922c2f577dfe84b140c4f840affd036c1a5feb1df20d21b6
Jul 17 17:25:22.949256 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.949100    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"]
Jul 17 17:25:22.973746 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.973663    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv" event={"ID":"3eccb27e-8b40-4ed3-a893-537bba16e278","Type":"ContainerStarted","Data":"e0e36627f1b3fc184edbbfa48ee4f632559613b5b360535c66fa79f35373722d"}
Jul 17 17:25:22.975095 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:22.975064    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst" event={"ID":"966fa4a8-a03e-41d1-a09e-177b9ae6542f","Type":"ContainerStarted","Data":"a8d16b8677eeb39f922c2f577dfe84b140c4f840affd036c1a5feb1df20d21b6"}
Jul 17 17:25:23.001392 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:23.001358    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d9xdg"]
Jul 17 17:25:23.004379 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:25:23.004349    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode98b193a_4711_4e94_b692_d0d36fe172d3.slice/crio-abcba0e0f30f2fcf1a11f59044181e8999cc4f3eaeea80cfbc07c7c64e823aa1 WatchSource:0}: Error finding container abcba0e0f30f2fcf1a11f59044181e8999cc4f3eaeea80cfbc07c7c64e823aa1: Status 404 returned error can't find the container with id abcba0e0f30f2fcf1a11f59044181e8999cc4f3eaeea80cfbc07c7c64e823aa1
Jul 17 17:25:23.042673 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:23.042643    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-jmwvg"]
Jul 17 17:25:23.045652 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:25:23.045618    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c455fa4_d71d_4748_abe2_6c903f6ae0f3.slice/crio-c8bea516b16871dfbe1b1cdb2117603eef917889a75d27a54e60f26447de4c68 WatchSource:0}: Error finding container c8bea516b16871dfbe1b1cdb2117603eef917889a75d27a54e60f26447de4c68: Status 404 returned error can't find the container with id c8bea516b16871dfbe1b1cdb2117603eef917889a75d27a54e60f26447de4c68
Jul 17 17:25:23.989890 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:23.989756    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-jmwvg" event={"ID":"3c455fa4-d71d-4748-abe2-6c903f6ae0f3","Type":"ContainerStarted","Data":"c8bea516b16871dfbe1b1cdb2117603eef917889a75d27a54e60f26447de4c68"}
Jul 17 17:25:23.994806 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:23.994768    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d9xdg" event={"ID":"e98b193a-4711-4e94-b692-d0d36fe172d3","Type":"ContainerStarted","Data":"abcba0e0f30f2fcf1a11f59044181e8999cc4f3eaeea80cfbc07c7c64e823aa1"}
Jul 17 17:25:25.956279 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:25.956238    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:25:25.959824 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:25.959796    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/642842c7-dd0b-4a6a-a358-eeea433a7d25-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-p8rl8\" (UID: \"642842c7-dd0b-4a6a-a358-eeea433a7d25\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:25:26.058334 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:26.057819    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:25:26.061233 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:26.061174    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/638d220b-0afb-46b6-8826-4110aeb4940b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-98j2x\" (UID: \"638d220b-0afb-46b6-8826-4110aeb4940b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:25:26.138669 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:26.138355    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"
Jul 17 17:25:26.230716 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:26.230628    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"
Jul 17 17:25:27.775396 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:27.775348    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:25:27.775839 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:27.775474    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:25:27.778696 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:27.778638    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"image-registry-85d4d7bb84-vg4h2\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") " pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:25:27.779460 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:27.779431    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de66d357-8e8a-4315-9db4-9adf5d86d420-metrics-tls\") pod \"dns-default-74tnw\" (UID: \"de66d357-8e8a-4315-9db4-9adf5d86d420\") " pod="openshift-dns/dns-default-74tnw"
Jul 17 17:25:27.782504 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:27.782462    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-smb8x\""
Jul 17 17:25:27.790184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:27.789909    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-74tnw"
Jul 17 17:25:28.067766 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:28.067730    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kbx6g\""
Jul 17 17:25:28.075984 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:28.075923    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:25:30.470465 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:30.470371    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x"]
Jul 17 17:25:31.055743 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:25:31.055705    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod638d220b_0afb_46b6_8826_4110aeb4940b.slice/crio-a35975fbc226f4d63e1239f97f29b7ab0e06c36eb0ebb487d66ca074c79022a3 WatchSource:0}: Error finding container a35975fbc226f4d63e1239f97f29b7ab0e06c36eb0ebb487d66ca074c79022a3: Status 404 returned error can't find the container with id a35975fbc226f4d63e1239f97f29b7ab0e06c36eb0ebb487d66ca074c79022a3
Jul 17 17:25:31.207532 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:31.207479    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8"]
Jul 17 17:25:31.558573 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:25:31.558532    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod642842c7_dd0b_4a6a_a358_eeea433a7d25.slice/crio-4dac69c66e0ad11f90dac031d327a77603e7a75aabd80c68a76f9e050bb9bbc3 WatchSource:0}: Error finding container 4dac69c66e0ad11f90dac031d327a77603e7a75aabd80c68a76f9e050bb9bbc3: Status 404 returned error can't find the container with id 4dac69c66e0ad11f90dac031d327a77603e7a75aabd80c68a76f9e050bb9bbc3
Jul 17 17:25:31.739215 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:31.738968    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85d4d7bb84-vg4h2"]
Jul 17 17:25:31.751116 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:31.751005    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-74tnw"]
Jul 17 17:25:31.755924 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:25:31.755893    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde66d357_8e8a_4315_9db4_9adf5d86d420.slice/crio-887956662b66675ea6b158ab8f3c402df0f4812b099af78ca3254c5326953922 WatchSource:0}: Error finding container 887956662b66675ea6b158ab8f3c402df0f4812b099af78ca3254c5326953922: Status 404 returned error can't find the container with id 887956662b66675ea6b158ab8f3c402df0f4812b099af78ca3254c5326953922
Jul 17 17:25:31.893300 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:31.893203    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2cr94"
Jul 17 17:25:32.037829 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.037787    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv" event={"ID":"3eccb27e-8b40-4ed3-a893-537bba16e278","Type":"ContainerStarted","Data":"8a1552eb7fcdc283587cee73e4020524dff0349f8aac87d1fff9ecab0a1eeed8"}
Jul 17 17:25:32.038468 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.038357    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:32.040424 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.040398    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv"
Jul 17 17:25:32.040791 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.040758    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2" event={"ID":"f3e18948-e8c7-4229-9f8b-7322ed643b9f","Type":"ContainerStarted","Data":"556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277"}
Jul 17 17:25:32.040885 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.040805    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2" event={"ID":"f3e18948-e8c7-4229-9f8b-7322ed643b9f","Type":"ContainerStarted","Data":"e39eb7d0c7e056f4e52ee62ba28c0a8fd9366ce676be301556857c1bc09e32cf"}
Jul 17 17:25:32.040885 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.040824    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:25:32.042540 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.042517    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x" event={"ID":"638d220b-0afb-46b6-8826-4110aeb4940b","Type":"ContainerStarted","Data":"a35975fbc226f4d63e1239f97f29b7ab0e06c36eb0ebb487d66ca074c79022a3"}
Jul 17 17:25:32.043968 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.043942    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8" event={"ID":"642842c7-dd0b-4a6a-a358-eeea433a7d25","Type":"ContainerStarted","Data":"4dac69c66e0ad11f90dac031d327a77603e7a75aabd80c68a76f9e050bb9bbc3"}
Jul 17 17:25:32.045897 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.045851    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst" event={"ID":"966fa4a8-a03e-41d1-a09e-177b9ae6542f","Type":"ContainerStarted","Data":"8958676a86753085fcd09a415516194bd58eb6f883fd0c36800dc90a88170797"}
Jul 17 17:25:32.047425 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.047404    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-74tnw" event={"ID":"de66d357-8e8a-4315-9db4-9adf5d86d420","Type":"ContainerStarted","Data":"887956662b66675ea6b158ab8f3c402df0f4812b099af78ca3254c5326953922"}
Jul 17 17:25:32.055940 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.055881    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc6d9484b-dbqhv" podStartSLOduration=1.445197211 podStartE2EDuration="10.055859174s" totalImagesPullingTime="8.610661963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:32.055159121 +0000 UTC m=+104.060514977" watchObservedRunningTime="2026-07-17 17:25:32.055859174 +0000 UTC m=+104.061215233"
Jul 17 17:25:32.089817 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.089151    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2" podStartSLOduration=81.089129793 podStartE2EDuration="1m21.089129793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:24:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:32.087233141 +0000 UTC m=+104.092588996" watchObservedRunningTime="2026-07-17 17:25:32.089129793 +0000 UTC m=+104.094485649"
Jul 17 17:25:32.103809 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:32.103737    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bcbc75fd-qqnst" podStartSLOduration=1.487461393 podStartE2EDuration="10.10370926s" totalImagesPullingTime="8.616247867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:32.101547429 +0000 UTC m=+104.106903283" watchObservedRunningTime="2026-07-17 17:25:32.10370926 +0000 UTC m=+104.109065112"
Jul 17 17:25:33.056678 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:33.056604    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d9xdg" event={"ID":"e98b193a-4711-4e94-b692-d0d36fe172d3","Type":"ContainerStarted","Data":"18a8b6ac0d2a59671c3206dc76848db37a1c9c0f04b7147a9a0ec3ae7df98650"}
Jul 17 17:25:45.107056 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:45.107017    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x" event={"ID":"638d220b-0afb-46b6-8826-4110aeb4940b","Type":"ContainerStarted","Data":"e62b64bf08e00911cd700b1c2da7a2064e7c2ac8f40335f836faf7206c3d5116"}
Jul 17 17:25:45.110960 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:45.110629    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8" event={"ID":"642842c7-dd0b-4a6a-a358-eeea433a7d25","Type":"ContainerStarted","Data":"ec7127e0ee88df5898ed6d6d36e6c5ffbb4d1ab301709c49a7bda8a355d9e7e2"}
Jul 17 17:25:45.123636 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:45.123574    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-98j2x" podStartSLOduration=45.260483701 podStartE2EDuration="51.123552452s" totalImagesPullingTime="5.863068751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:45.122984002 +0000 UTC m=+117.128339856" watchObservedRunningTime="2026-07-17 17:25:45.123552452 +0000 UTC m=+117.128908306"
Jul 17 17:25:45.133604 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:45.133570    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85d4d7bb84-vg4h2"]
Jul 17 17:25:45.142289 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:45.142230    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-p8rl8" podStartSLOduration=37.779746416 podStartE2EDuration="51.142207356s" totalImagesPullingTime="13.36246094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:45.141792603 +0000 UTC m=+117.147148459" watchObservedRunningTime="2026-07-17 17:25:45.142207356 +0000 UTC m=+117.147563212"
Jul 17 17:25:46.123888 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:46.123794    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d9xdg" event={"ID":"e98b193a-4711-4e94-b692-d0d36fe172d3","Type":"ContainerStarted","Data":"8944cb41cb15e8e23c1cea0a7c8085237526eb095da01e69e3a02badef773c7d"}
Jul 17 17:25:46.125602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:46.125530    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-jmwvg" event={"ID":"3c455fa4-d71d-4748-abe2-6c903f6ae0f3","Type":"ContainerStarted","Data":"690448ef8135e4cfee70c92a2110fd0fc6d2dbb540aa7ba256e237ccce0b624a"}
Jul 17 17:25:46.126947 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:46.126620    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-jmwvg"
Jul 17 17:25:46.130623 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:46.130115    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-74tnw" event={"ID":"de66d357-8e8a-4315-9db4-9adf5d86d420","Type":"ContainerStarted","Data":"2dfd7ee36bef58a8fb00649045709c5f4d69033c44b40ea365d2fbab09d79019"}
Jul 17 17:25:46.130623 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:46.130182    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-74tnw"
Jul 17 17:25:46.130623 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:46.130210    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-74tnw" event={"ID":"de66d357-8e8a-4315-9db4-9adf5d86d420","Type":"ContainerStarted","Data":"8b83032dee620e019bbaec3f63df91cc3a38f6d2c4a3396624e6aedf07eec93a"}
Jul 17 17:25:46.144142 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:46.144065    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-jmwvg"
Jul 17 17:25:46.161962 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:46.161890    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-74tnw" podStartSLOduration=69.966742642 podStartE2EDuration="1m23.161856457s" totalImagesPullingTime="13.195113815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:24:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:46.159190688 +0000 UTC m=+118.164546569" watchObservedRunningTime="2026-07-17 17:25:46.161856457 +0000 UTC m=+118.167212311"
Jul 17 17:25:46.169330 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:46.163353    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-jmwvg" podStartSLOduration=2.208769934 podStartE2EDuration="24.163329577s" totalImagesPullingTime="21.954559643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:46.143173002 +0000 UTC m=+118.148528859" watchObservedRunningTime="2026-07-17 17:25:46.163329577 +0000 UTC m=+118.168685431"
Jul 17 17:25:51.155708 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:51.155668    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d9xdg" event={"ID":"e98b193a-4711-4e94-b692-d0d36fe172d3","Type":"ContainerStarted","Data":"cf463f18be98debf8edcd2a323bc863eb70c6ce253054ffbfb67d1cfb816ae04"}
Jul 17 17:25:51.176566 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:51.176475    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d9xdg" podStartSLOduration=2.20447157 podStartE2EDuration="29.176455586s" totalImagesPullingTime="26.971984016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:25:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:25:51.175142061 +0000 UTC m=+123.180497934" watchObservedRunningTime="2026-07-17 17:25:51.176455586 +0000 UTC m=+123.181811441"
Jul 17 17:25:55.140383 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:55.140349    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:25:56.136759 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:56.136728    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-74tnw"
Jul 17 17:25:57.173570 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.173529    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g24rs"]
Jul 17 17:25:57.294912 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.294876    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.298238 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.298213    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 17:25:57.298401 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.298213    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 17:25:57.298401 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.298213    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 17:25:57.299131 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.299107    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2zmsl\""
Jul 17 17:25:57.299250 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.299234    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 17:25:57.359512 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.359454    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-accelerators-collector-config\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.359704 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.359534    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.359704 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.359576    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-tls\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.359704 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.359673    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:25:57.359865 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.359780    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/450ae286-b9a8-4e77-a616-46040d067be1-sys\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.359865 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.359837    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/450ae286-b9a8-4e77-a616-46040d067be1-metrics-client-ca\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.359968 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.359878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45cs2\" (UniqueName: \"kubernetes.io/projected/450ae286-b9a8-4e77-a616-46040d067be1-kube-api-access-45cs2\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.359968 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.359932    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-textfile\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.360068 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.359973    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-wtmp\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.360068 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.360002    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/450ae286-b9a8-4e77-a616-46040d067be1-root\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.362484 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.362416    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d-metrics-certs\") pod \"network-metrics-daemon-zvc9x\" (UID: \"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d\") " pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:25:57.461454 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461353    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-wtmp\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461454 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461396    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/450ae286-b9a8-4e77-a616-46040d067be1-root\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461454 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461422    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-accelerators-collector-config\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461526    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/450ae286-b9a8-4e77-a616-46040d067be1-root\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461552    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461576    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-wtmp\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461591    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-tls\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461662    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/450ae286-b9a8-4e77-a616-46040d067be1-sys\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461689    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/450ae286-b9a8-4e77-a616-46040d067be1-metrics-client-ca\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461715    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45cs2\" (UniqueName: \"kubernetes.io/projected/450ae286-b9a8-4e77-a616-46040d067be1-kube-api-access-45cs2\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461757    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/450ae286-b9a8-4e77-a616-46040d067be1-sys\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.461767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.461758    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-textfile\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.462227 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.462024    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-textfile\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.462227 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.462097    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-accelerators-collector-config\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.462407 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.462390    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/450ae286-b9a8-4e77-a616-46040d067be1-metrics-client-ca\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.463951 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.463929    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-tls\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.464046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.463973    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/450ae286-b9a8-4e77-a616-46040d067be1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.471723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.471694    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45cs2\" (UniqueName: \"kubernetes.io/projected/450ae286-b9a8-4e77-a616-46040d067be1-kube-api-access-45cs2\") pod \"node-exporter-g24rs\" (UID: \"450ae286-b9a8-4e77-a616-46040d067be1\") " pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.605547 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.605511    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g24rs"
Jul 17 17:25:57.626092 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.626061    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nlczc\""
Jul 17 17:25:57.634568 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.634537    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvc9x"
Jul 17 17:25:57.647644 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:25:57.647619    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450ae286_b9a8_4e77_a616_46040d067be1.slice/crio-0f07ac5272a34974817552d179de59ef76b3a7b65651b6264da39889799f2546 WatchSource:0}: Error finding container 0f07ac5272a34974817552d179de59ef76b3a7b65651b6264da39889799f2546: Status 404 returned error can't find the container with id 0f07ac5272a34974817552d179de59ef76b3a7b65651b6264da39889799f2546
Jul 17 17:25:57.794329 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:57.794299    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zvc9x"]
Jul 17 17:25:57.796864 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:25:57.796836    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d4bd31a_c3d5_41e8_a375_b6a1f519eb2d.slice/crio-124b40a56178f216b8b376d51a067d9174d8536adfb723b00b8620002d17eb58 WatchSource:0}: Error finding container 124b40a56178f216b8b376d51a067d9174d8536adfb723b00b8620002d17eb58: Status 404 returned error can't find the container with id 124b40a56178f216b8b376d51a067d9174d8536adfb723b00b8620002d17eb58
Jul 17 17:25:58.187273 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:58.187235    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g24rs" event={"ID":"450ae286-b9a8-4e77-a616-46040d067be1","Type":"ContainerStarted","Data":"0f07ac5272a34974817552d179de59ef76b3a7b65651b6264da39889799f2546"}
Jul 17 17:25:58.188723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:25:58.188687    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zvc9x" event={"ID":"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d","Type":"ContainerStarted","Data":"124b40a56178f216b8b376d51a067d9174d8536adfb723b00b8620002d17eb58"}
Jul 17 17:26:00.198021 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:00.197983    2572 generic.go:356] "Generic (PLEG): container finished" podID="450ae286-b9a8-4e77-a616-46040d067be1" containerID="9715de5addb0944f1841e60794bd30a48ef0855efd5dab1783f8b3cd21cf24d9" exitCode=0
Jul 17 17:26:00.198506 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:00.198049    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g24rs" event={"ID":"450ae286-b9a8-4e77-a616-46040d067be1","Type":"ContainerDied","Data":"9715de5addb0944f1841e60794bd30a48ef0855efd5dab1783f8b3cd21cf24d9"}
Jul 17 17:26:00.200087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:00.200064    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zvc9x" event={"ID":"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d","Type":"ContainerStarted","Data":"653d381bd1cbd4331c4c7475c9a0a9c4fb00a6d57b3c38a3ff2dbf08bf8841fa"}
Jul 17 17:26:00.200164 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:00.200116    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zvc9x" event={"ID":"8d4bd31a-c3d5-41e8-a375-b6a1f519eb2d","Type":"ContainerStarted","Data":"d20fd0bb7a3b6e7e1de9496b8545a85c17cff79481d61dac4511b5182d065d77"}
Jul 17 17:26:00.232292 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:00.232227    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zvc9x" podStartSLOduration=129.455666903 podStartE2EDuration="2m11.232209599s" totalImagesPullingTime="1.776542696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:26:00.231445164 +0000 UTC m=+132.236801019" watchObservedRunningTime="2026-07-17 17:26:00.232209599 +0000 UTC m=+132.237565452"
Jul 17 17:26:01.206211 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:01.206169    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g24rs" event={"ID":"450ae286-b9a8-4e77-a616-46040d067be1","Type":"ContainerStarted","Data":"9982a4aabf69cda14314f8545522ae10fedf60d723dc3b201840d0ae7464d140"}
Jul 17 17:26:01.206674 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:01.206240    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g24rs" event={"ID":"450ae286-b9a8-4e77-a616-46040d067be1","Type":"ContainerStarted","Data":"960f58b8ec1e455fd0f7a1242ebbe870df55cdc9c535a0ada51fd763adcee4dc"}
Jul 17 17:26:01.228243 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:01.228188    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g24rs" podStartSLOduration=2.188366663 podStartE2EDuration="4.228169699s" totalImagesPullingTime="1.923614703s" totalInitContainerRuntime="116.188333ms" isStatefulPod=true podCreationTimestamp="2026-07-17 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:26:01.226102896 +0000 UTC m=+133.231458776" watchObservedRunningTime="2026-07-17 17:26:01.228169699 +0000 UTC m=+133.233525596"
Jul 17 17:26:10.161266 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.161194    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2" podUID="f3e18948-e8c7-4229-9f8b-7322ed643b9f" containerName="registry" containerID="cri-o://556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277" gracePeriod=30
Jul 17 17:26:10.431852 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.431772    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:26:10.461415 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.461382    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-trusted-ca\") pod \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") "
Jul 17 17:26:10.461639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.461438    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-installation-pull-secrets\") pod \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") "
Jul 17 17:26:10.461639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.461476    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3e18948-e8c7-4229-9f8b-7322ed643b9f-ca-trust-extracted\") pod \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") "
Jul 17 17:26:10.461639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.461527    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-bound-sa-token\") pod \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") "
Jul 17 17:26:10.461639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.461563    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfsk5\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-kube-api-access-mfsk5\") pod \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") "
Jul 17 17:26:10.461639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.461592    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-image-registry-private-configuration\") pod \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") "
Jul 17 17:26:10.461639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.461621    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") pod \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") "
Jul 17 17:26:10.461945 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.461647    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-certificates\") pod \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\" (UID: \"f3e18948-e8c7-4229-9f8b-7322ed643b9f\") "
Jul 17 17:26:10.461945 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.461923    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f3e18948-e8c7-4229-9f8b-7322ed643b9f" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:26:10.462299 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.462211    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f3e18948-e8c7-4229-9f8b-7322ed643b9f" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:26:10.464606 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.464577    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f3e18948-e8c7-4229-9f8b-7322ed643b9f" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 17:26:10.464730 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.464691    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f3e18948-e8c7-4229-9f8b-7322ed643b9f" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:26:10.464799 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.464736    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f3e18948-e8c7-4229-9f8b-7322ed643b9f" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:26:10.464845 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.464796    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f3e18948-e8c7-4229-9f8b-7322ed643b9f" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 17:26:10.464880 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.464850    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-kube-api-access-mfsk5" (OuterVolumeSpecName: "kube-api-access-mfsk5") pod "f3e18948-e8c7-4229-9f8b-7322ed643b9f" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f"). InnerVolumeSpecName "kube-api-access-mfsk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:26:10.473289 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.473224    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3e18948-e8c7-4229-9f8b-7322ed643b9f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f3e18948-e8c7-4229-9f8b-7322ed643b9f" (UID: "f3e18948-e8c7-4229-9f8b-7322ed643b9f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:26:10.563263 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.563222    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-trusted-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:26:10.563630 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.563598    2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-installation-pull-secrets\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:26:10.563630 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.563631    2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3e18948-e8c7-4229-9f8b-7322ed643b9f-ca-trust-extracted\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:26:10.563853 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.563648    2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-bound-sa-token\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:26:10.563853 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.563663    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfsk5\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-kube-api-access-mfsk5\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:26:10.563853 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.563679    2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3e18948-e8c7-4229-9f8b-7322ed643b9f-image-registry-private-configuration\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:26:10.563853 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.563694    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-tls\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:26:10.563853 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:10.563709    2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3e18948-e8c7-4229-9f8b-7322ed643b9f-registry-certificates\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:26:11.242485 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:11.242439    2572 generic.go:356] "Generic (PLEG): container finished" podID="f3e18948-e8c7-4229-9f8b-7322ed643b9f" containerID="556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277" exitCode=0
Jul 17 17:26:11.242933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:11.242519    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2" event={"ID":"f3e18948-e8c7-4229-9f8b-7322ed643b9f","Type":"ContainerDied","Data":"556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277"}
Jul 17 17:26:11.242933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:11.242529    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2"
Jul 17 17:26:11.242933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:11.242562    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85d4d7bb84-vg4h2" event={"ID":"f3e18948-e8c7-4229-9f8b-7322ed643b9f","Type":"ContainerDied","Data":"e39eb7d0c7e056f4e52ee62ba28c0a8fd9366ce676be301556857c1bc09e32cf"}
Jul 17 17:26:11.242933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:11.242582    2572 scope.go:117] "RemoveContainer" containerID="556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277"
Jul 17 17:26:11.250607 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:11.250586    2572 scope.go:117] "RemoveContainer" containerID="556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277"
Jul 17 17:26:11.250871 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:26:11.250847    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277\": container with ID starting with 556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277 not found: ID does not exist" containerID="556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277"
Jul 17 17:26:11.250923 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:11.250880    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277"} err="failed to get container status \"556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277\": rpc error: code = NotFound desc = could not find container \"556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277\": container with ID starting with 556c507350042637913f62484db3324727fde20513dd704cbfe678df1c800277 not found: ID does not exist"
Jul 17 17:26:11.265445 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:11.265406    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85d4d7bb84-vg4h2"]
Jul 17 17:26:11.273105 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:11.273071    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85d4d7bb84-vg4h2"]
Jul 17 17:26:12.516124 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:12.516092    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3e18948-e8c7-4229-9f8b-7322ed643b9f" path="/var/lib/kubelet/pods/f3e18948-e8c7-4229-9f8b-7322ed643b9f/volumes"
Jul 17 17:26:21.281471 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:21.281377    2572 generic.go:356] "Generic (PLEG): container finished" podID="bcf3189c-2328-4a13-85f6-0cb7423020cc" containerID="a18735a4b72f1b8559a6f7b02a1ffa21f732954458ca528f81858b58f78759ce" exitCode=0
Jul 17 17:26:21.281471 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:21.281451    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-kgqwx" event={"ID":"bcf3189c-2328-4a13-85f6-0cb7423020cc","Type":"ContainerDied","Data":"a18735a4b72f1b8559a6f7b02a1ffa21f732954458ca528f81858b58f78759ce"}
Jul 17 17:26:21.281877 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:21.281860    2572 scope.go:117] "RemoveContainer" containerID="a18735a4b72f1b8559a6f7b02a1ffa21f732954458ca528f81858b58f78759ce"
Jul 17 17:26:22.139759 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:22.139726    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-djgz7_f15bcab1-4a49-45b7-94a7-f7175f8ce2c5/serve-healthcheck-canary/0.log"
Jul 17 17:26:22.286840 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:26:22.286805    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-kgqwx" event={"ID":"bcf3189c-2328-4a13-85f6-0cb7423020cc","Type":"ContainerStarted","Data":"332ff59b9cd39f6ee2624097dac468c82493a69a55059da461ec59758437d917"}
Jul 17 17:28:39.145954 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.145919    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"]
Jul 17 17:28:39.146384 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.146185    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3e18948-e8c7-4229-9f8b-7322ed643b9f" containerName="registry"
Jul 17 17:28:39.146384 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.146196    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3e18948-e8c7-4229-9f8b-7322ed643b9f" containerName="registry"
Jul 17 17:28:39.146384 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.146255    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3e18948-e8c7-4229-9f8b-7322ed643b9f" containerName="registry"
Jul 17 17:28:39.148155 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.148138    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.150850 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.150824    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 17 17:28:39.150988 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.150859    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 17 17:28:39.151814 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.151796    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pq2bx\""
Jul 17 17:28:39.157016 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.156992    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"]
Jul 17 17:28:39.230544 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.230475    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk7j5\" (UniqueName: \"kubernetes.io/projected/8dfe29ec-bf6f-46a6-8308-5654188759a5-kube-api-access-wk7j5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.230740 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.230555    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.230740 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.230648    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.331644 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.331600    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.331644 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.331648    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk7j5\" (UniqueName: \"kubernetes.io/projected/8dfe29ec-bf6f-46a6-8308-5654188759a5-kube-api-access-wk7j5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.331853 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.331673    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.332018 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.331996    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.332059 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.332020    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.340617 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.340585    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk7j5\" (UniqueName: \"kubernetes.io/projected/8dfe29ec-bf6f-46a6-8308-5654188759a5-kube-api-access-wk7j5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.457968 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.457874    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:39.592083 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.592055    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"]
Jul 17 17:28:39.594780 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:28:39.594748    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dfe29ec_bf6f_46a6_8308_5654188759a5.slice/crio-0a2c4b45f6f4bdd4043f833743c9fa74e6e5d37461f86ef6a8d949314cdfb311 WatchSource:0}: Error finding container 0a2c4b45f6f4bdd4043f833743c9fa74e6e5d37461f86ef6a8d949314cdfb311: Status 404 returned error can't find the container with id 0a2c4b45f6f4bdd4043f833743c9fa74e6e5d37461f86ef6a8d949314cdfb311
Jul 17 17:28:39.784235 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:39.784140    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w" event={"ID":"8dfe29ec-bf6f-46a6-8308-5654188759a5","Type":"ContainerStarted","Data":"0a2c4b45f6f4bdd4043f833743c9fa74e6e5d37461f86ef6a8d949314cdfb311"}
Jul 17 17:28:45.810620 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:45.810588    2572 generic.go:356] "Generic (PLEG): container finished" podID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerID="f05542c718b7352ad53fd145af4a0253a9b2ef3c429b0218a22d2810c54fa393" exitCode=0
Jul 17 17:28:45.811010 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:45.810630    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w" event={"ID":"8dfe29ec-bf6f-46a6-8308-5654188759a5","Type":"ContainerDied","Data":"f05542c718b7352ad53fd145af4a0253a9b2ef3c429b0218a22d2810c54fa393"}
Jul 17 17:28:48.421840 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:48.421801    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:28:48.427885 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:48.427854    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:28:48.429161 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:48.429135    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:28:48.435350 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:48.435314    2572 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 17:28:48.437187 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:48.437166    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:28:48.825117 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:48.825078    2572 generic.go:356] "Generic (PLEG): container finished" podID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerID="77178aa759dc4406d2d5ce2fcfd7f9a19d770c864eaf75b0b508bd1a437bf22f" exitCode=0
Jul 17 17:28:48.826472 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:48.825127    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w" event={"ID":"8dfe29ec-bf6f-46a6-8308-5654188759a5","Type":"ContainerDied","Data":"77178aa759dc4406d2d5ce2fcfd7f9a19d770c864eaf75b0b508bd1a437bf22f"}
Jul 17 17:28:55.856329 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:55.856294    2572 generic.go:356] "Generic (PLEG): container finished" podID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerID="17cf37ede17d80c976f5ed1c3f5afde5144bbe740a94e6d2b7095f926549e078" exitCode=0
Jul 17 17:28:55.856721 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:55.856377    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w" event={"ID":"8dfe29ec-bf6f-46a6-8308-5654188759a5","Type":"ContainerDied","Data":"17cf37ede17d80c976f5ed1c3f5afde5144bbe740a94e6d2b7095f926549e078"}
Jul 17 17:28:56.990123 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:56.990077    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:57.084677 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.084642    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk7j5\" (UniqueName: \"kubernetes.io/projected/8dfe29ec-bf6f-46a6-8308-5654188759a5-kube-api-access-wk7j5\") pod \"8dfe29ec-bf6f-46a6-8308-5654188759a5\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") "
Jul 17 17:28:57.084866 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.084754    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-util\") pod \"8dfe29ec-bf6f-46a6-8308-5654188759a5\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") "
Jul 17 17:28:57.084866 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.084774    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-bundle\") pod \"8dfe29ec-bf6f-46a6-8308-5654188759a5\" (UID: \"8dfe29ec-bf6f-46a6-8308-5654188759a5\") "
Jul 17 17:28:57.085461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.085434    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-bundle" (OuterVolumeSpecName: "bundle") pod "8dfe29ec-bf6f-46a6-8308-5654188759a5" (UID: "8dfe29ec-bf6f-46a6-8308-5654188759a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:28:57.086923 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.086902    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dfe29ec-bf6f-46a6-8308-5654188759a5-kube-api-access-wk7j5" (OuterVolumeSpecName: "kube-api-access-wk7j5") pod "8dfe29ec-bf6f-46a6-8308-5654188759a5" (UID: "8dfe29ec-bf6f-46a6-8308-5654188759a5"). InnerVolumeSpecName "kube-api-access-wk7j5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:28:57.088745 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.088719    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-util" (OuterVolumeSpecName: "util") pod "8dfe29ec-bf6f-46a6-8308-5654188759a5" (UID: "8dfe29ec-bf6f-46a6-8308-5654188759a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:28:57.185597 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.185467    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-util\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:28:57.185597 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.185530    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dfe29ec-bf6f-46a6-8308-5654188759a5-bundle\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:28:57.185597 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.185543    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk7j5\" (UniqueName: \"kubernetes.io/projected/8dfe29ec-bf6f-46a6-8308-5654188759a5-kube-api-access-wk7j5\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:28:57.865421 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.865390    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w"
Jul 17 17:28:57.865421 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.865400    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402lj4w" event={"ID":"8dfe29ec-bf6f-46a6-8308-5654188759a5","Type":"ContainerDied","Data":"0a2c4b45f6f4bdd4043f833743c9fa74e6e5d37461f86ef6a8d949314cdfb311"}
Jul 17 17:28:57.865714 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:28:57.865442    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a2c4b45f6f4bdd4043f833743c9fa74e6e5d37461f86ef6a8d949314cdfb311"
Jul 17 17:29:02.060793 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.060760    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"]
Jul 17 17:29:02.061305 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.061024    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerName="util"
Jul 17 17:29:02.061305 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.061034    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerName="util"
Jul 17 17:29:02.061305 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.061053    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerName="pull"
Jul 17 17:29:02.061305 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.061058    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerName="pull"
Jul 17 17:29:02.061305 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.061069    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerName="extract"
Jul 17 17:29:02.061305 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.061074    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerName="extract"
Jul 17 17:29:02.061305 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.061120    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dfe29ec-bf6f-46a6-8308-5654188759a5" containerName="extract"
Jul 17 17:29:02.102544 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.102471    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"]
Jul 17 17:29:02.102544 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.102521    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"
Jul 17 17:29:02.105203 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.105179    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 17 17:29:02.105304 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.105209    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 17 17:29:02.105304 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.105244    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-pkqcp\""
Jul 17 17:29:02.224797 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.224761    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ff29822-7650-467d-aa94-5448cce488fc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9xcqc\" (UID: \"4ff29822-7650-467d-aa94-5448cce488fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"
Jul 17 17:29:02.224797 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.224802    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqbtv\" (UniqueName: \"kubernetes.io/projected/4ff29822-7650-467d-aa94-5448cce488fc-kube-api-access-sqbtv\") pod \"cert-manager-operator-controller-manager-55c444bf86-9xcqc\" (UID: \"4ff29822-7650-467d-aa94-5448cce488fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"
Jul 17 17:29:02.325209 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.325127    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ff29822-7650-467d-aa94-5448cce488fc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9xcqc\" (UID: \"4ff29822-7650-467d-aa94-5448cce488fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"
Jul 17 17:29:02.325209 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.325167    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqbtv\" (UniqueName: \"kubernetes.io/projected/4ff29822-7650-467d-aa94-5448cce488fc-kube-api-access-sqbtv\") pod \"cert-manager-operator-controller-manager-55c444bf86-9xcqc\" (UID: \"4ff29822-7650-467d-aa94-5448cce488fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"
Jul 17 17:29:02.325551 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.325530    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ff29822-7650-467d-aa94-5448cce488fc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9xcqc\" (UID: \"4ff29822-7650-467d-aa94-5448cce488fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"
Jul 17 17:29:02.336650 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.336617    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqbtv\" (UniqueName: \"kubernetes.io/projected/4ff29822-7650-467d-aa94-5448cce488fc-kube-api-access-sqbtv\") pod \"cert-manager-operator-controller-manager-55c444bf86-9xcqc\" (UID: \"4ff29822-7650-467d-aa94-5448cce488fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"
Jul 17 17:29:02.411886 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.411849    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"
Jul 17 17:29:02.573234 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.573202    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc"]
Jul 17 17:29:02.576779 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:29:02.576698    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff29822_7650_467d_aa94_5448cce488fc.slice/crio-546505a806fe792f4967bf5b1846750f484bbf5cec49baab9eec7d866368b66b WatchSource:0}: Error finding container 546505a806fe792f4967bf5b1846750f484bbf5cec49baab9eec7d866368b66b: Status 404 returned error can't find the container with id 546505a806fe792f4967bf5b1846750f484bbf5cec49baab9eec7d866368b66b
Jul 17 17:29:02.580237 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.580220    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 17:29:02.886877 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:02.886788    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc" event={"ID":"4ff29822-7650-467d-aa94-5448cce488fc","Type":"ContainerStarted","Data":"546505a806fe792f4967bf5b1846750f484bbf5cec49baab9eec7d866368b66b"}
Jul 17 17:29:05.901900 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:05.901858    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc" event={"ID":"4ff29822-7650-467d-aa94-5448cce488fc","Type":"ContainerStarted","Data":"a43d146ce36b2becf5637ebaf6a4af9a653e7784f6b0d53961a2c28f9c5e6800"}
Jul 17 17:29:05.929367 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:05.929309    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9xcqc" podStartSLOduration=1.633499827 podStartE2EDuration="3.929292552s" totalImagesPullingTime="2.295792725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:29:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:29:05.92852107 +0000 UTC m=+317.933876926" watchObservedRunningTime="2026-07-17 17:29:05.929292552 +0000 UTC m=+317.934648405"
Jul 17 17:29:06.226896 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.226811    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cckv8"]
Jul 17 17:29:06.249795 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.249762    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cckv8"]
Jul 17 17:29:06.249988 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.249907    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:06.252975 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.252938    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 17 17:29:06.253306 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.253285    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7bbx8\""
Jul 17 17:29:06.253406 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.253379    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 17 17:29:06.359549 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.359482    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8902360-2c91-43f1-ae3b-132429db8211-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cckv8\" (UID: \"a8902360-2c91-43f1-ae3b-132429db8211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:06.359734 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.359587    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k77s9\" (UniqueName: \"kubernetes.io/projected/a8902360-2c91-43f1-ae3b-132429db8211-kube-api-access-k77s9\") pod \"cert-manager-webhook-6486ffd997-cckv8\" (UID: \"a8902360-2c91-43f1-ae3b-132429db8211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:06.457395 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.457362    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"]
Jul 17 17:29:06.460629 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.460602    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8902360-2c91-43f1-ae3b-132429db8211-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cckv8\" (UID: \"a8902360-2c91-43f1-ae3b-132429db8211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:06.460767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.460651    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k77s9\" (UniqueName: \"kubernetes.io/projected/a8902360-2c91-43f1-ae3b-132429db8211-kube-api-access-k77s9\") pod \"cert-manager-webhook-6486ffd997-cckv8\" (UID: \"a8902360-2c91-43f1-ae3b-132429db8211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:06.477551 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.477448    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"]
Jul 17 17:29:06.477717 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.477598    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.479665 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.479635    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8902360-2c91-43f1-ae3b-132429db8211-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cckv8\" (UID: \"a8902360-2c91-43f1-ae3b-132429db8211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:06.480135 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.480104    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 17 17:29:06.480288 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.480237    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 17 17:29:06.480352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.480335    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pq2bx\""
Jul 17 17:29:06.480455 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.480431    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k77s9\" (UniqueName: \"kubernetes.io/projected/a8902360-2c91-43f1-ae3b-132429db8211-kube-api-access-k77s9\") pod \"cert-manager-webhook-6486ffd997-cckv8\" (UID: \"a8902360-2c91-43f1-ae3b-132429db8211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:06.559983 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.559946    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:06.561210 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.561184    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p46h\" (UniqueName: \"kubernetes.io/projected/79604c15-25ef-4898-8844-70d00948fb77-kube-api-access-8p46h\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.561326 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.561223    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.561326 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.561313    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.662222 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.662184    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p46h\" (UniqueName: \"kubernetes.io/projected/79604c15-25ef-4898-8844-70d00948fb77-kube-api-access-8p46h\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.662395 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.662229    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.662395 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.662290    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.662726 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.662708    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.662826 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.662802    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.684071 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.684031    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p46h\" (UniqueName: \"kubernetes.io/projected/79604c15-25ef-4898-8844-70d00948fb77-kube-api-access-8p46h\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.738229 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.738201    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cckv8"]
Jul 17 17:29:06.740140 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:29:06.740111    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8902360_2c91_43f1_ae3b_132429db8211.slice/crio-3f91959e0f39e02c2f338d1b4286186271177366351592feadb05afc4a47f43a WatchSource:0}: Error finding container 3f91959e0f39e02c2f338d1b4286186271177366351592feadb05afc4a47f43a: Status 404 returned error can't find the container with id 3f91959e0f39e02c2f338d1b4286186271177366351592feadb05afc4a47f43a
Jul 17 17:29:06.793325 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.793292    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:06.908572 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.908530    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8" event={"ID":"a8902360-2c91-43f1-ae3b-132429db8211","Type":"ContainerStarted","Data":"3f91959e0f39e02c2f338d1b4286186271177366351592feadb05afc4a47f43a"}
Jul 17 17:29:06.964060 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:06.964034    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"]
Jul 17 17:29:06.966459 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:29:06.966416    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79604c15_25ef_4898_8844_70d00948fb77.slice/crio-1dec786bf20373362de6d87ff7cbc503f019a645610e196af5f6c64d439ca3f3 WatchSource:0}: Error finding container 1dec786bf20373362de6d87ff7cbc503f019a645610e196af5f6c64d439ca3f3: Status 404 returned error can't find the container with id 1dec786bf20373362de6d87ff7cbc503f019a645610e196af5f6c64d439ca3f3
Jul 17 17:29:07.916082 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:07.916045    2572 generic.go:356] "Generic (PLEG): container finished" podID="79604c15-25ef-4898-8844-70d00948fb77" containerID="b55e57c4da84cd0ba2a31111e16082ae3ca1831dd4bdb156c135b0cb6cfb5d1a" exitCode=0
Jul 17 17:29:07.916708 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:07.916127    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm" event={"ID":"79604c15-25ef-4898-8844-70d00948fb77","Type":"ContainerDied","Data":"b55e57c4da84cd0ba2a31111e16082ae3ca1831dd4bdb156c135b0cb6cfb5d1a"}
Jul 17 17:29:07.916708 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:07.916186    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm" event={"ID":"79604c15-25ef-4898-8844-70d00948fb77","Type":"ContainerStarted","Data":"1dec786bf20373362de6d87ff7cbc503f019a645610e196af5f6c64d439ca3f3"}
Jul 17 17:29:10.931060 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:10.931023    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8" event={"ID":"a8902360-2c91-43f1-ae3b-132429db8211","Type":"ContainerStarted","Data":"2814e21843da20e4c26f6f00d55175b9e13e8f2929cefcbcbe3cc6d81212d7d6"}
Jul 17 17:29:10.931590 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:10.931096    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:10.932809 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:10.932781    2572 generic.go:356] "Generic (PLEG): container finished" podID="79604c15-25ef-4898-8844-70d00948fb77" containerID="cd5d098d4c096fc63b40cb94b9b5f344ef12c35f26193c0ae2660586073c32cc" exitCode=0
Jul 17 17:29:10.932897 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:10.932836    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm" event={"ID":"79604c15-25ef-4898-8844-70d00948fb77","Type":"ContainerDied","Data":"cd5d098d4c096fc63b40cb94b9b5f344ef12c35f26193c0ae2660586073c32cc"}
Jul 17 17:29:10.950167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:10.950112    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8" podStartSLOduration=1.3758835710000001 podStartE2EDuration="4.950095466s" totalImagesPullingTime="3.574211895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:29:10.949252807 +0000 UTC m=+322.954608663" watchObservedRunningTime="2026-07-17 17:29:10.950095466 +0000 UTC m=+322.955451320"
Jul 17 17:29:11.939371 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:11.939336    2572 generic.go:356] "Generic (PLEG): container finished" podID="79604c15-25ef-4898-8844-70d00948fb77" containerID="19fbc0013a3a1dfd90688075bded72031473745cf3816a3057cbcb25a49c248b" exitCode=0
Jul 17 17:29:11.939769 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:11.939410    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm" event={"ID":"79604c15-25ef-4898-8844-70d00948fb77","Type":"ContainerDied","Data":"19fbc0013a3a1dfd90688075bded72031473745cf3816a3057cbcb25a49c248b"}
Jul 17 17:29:13.078151 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.078121    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:13.222043 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.221945    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p46h\" (UniqueName: \"kubernetes.io/projected/79604c15-25ef-4898-8844-70d00948fb77-kube-api-access-8p46h\") pod \"79604c15-25ef-4898-8844-70d00948fb77\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") "
Jul 17 17:29:13.222043 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.222016    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-util\") pod \"79604c15-25ef-4898-8844-70d00948fb77\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") "
Jul 17 17:29:13.222248 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.222115    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-bundle\") pod \"79604c15-25ef-4898-8844-70d00948fb77\" (UID: \"79604c15-25ef-4898-8844-70d00948fb77\") "
Jul 17 17:29:13.222507 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.222470    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-bundle" (OuterVolumeSpecName: "bundle") pod "79604c15-25ef-4898-8844-70d00948fb77" (UID: "79604c15-25ef-4898-8844-70d00948fb77"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:29:13.224329 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.224304    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79604c15-25ef-4898-8844-70d00948fb77-kube-api-access-8p46h" (OuterVolumeSpecName: "kube-api-access-8p46h") pod "79604c15-25ef-4898-8844-70d00948fb77" (UID: "79604c15-25ef-4898-8844-70d00948fb77"). InnerVolumeSpecName "kube-api-access-8p46h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:29:13.226222 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.226198    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-util" (OuterVolumeSpecName: "util") pod "79604c15-25ef-4898-8844-70d00948fb77" (UID: "79604c15-25ef-4898-8844-70d00948fb77"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:29:13.322795 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.322756    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-bundle\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:29:13.322795 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.322790    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p46h\" (UniqueName: \"kubernetes.io/projected/79604c15-25ef-4898-8844-70d00948fb77-kube-api-access-8p46h\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:29:13.322795 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.322802    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79604c15-25ef-4898-8844-70d00948fb77-util\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:29:13.949946 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.949914    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm"
Jul 17 17:29:13.950128 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.949942    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5s9xm" event={"ID":"79604c15-25ef-4898-8844-70d00948fb77","Type":"ContainerDied","Data":"1dec786bf20373362de6d87ff7cbc503f019a645610e196af5f6c64d439ca3f3"}
Jul 17 17:29:13.950128 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:13.949990    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dec786bf20373362de6d87ff7cbc503f019a645610e196af5f6c64d439ca3f3"
Jul 17 17:29:16.733160 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.733124    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kpxzl"]
Jul 17 17:29:16.733578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.733392    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79604c15-25ef-4898-8844-70d00948fb77" containerName="util"
Jul 17 17:29:16.733578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.733404    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="79604c15-25ef-4898-8844-70d00948fb77" containerName="util"
Jul 17 17:29:16.733578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.733417    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79604c15-25ef-4898-8844-70d00948fb77" containerName="pull"
Jul 17 17:29:16.733578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.733423    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="79604c15-25ef-4898-8844-70d00948fb77" containerName="pull"
Jul 17 17:29:16.733578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.733432    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79604c15-25ef-4898-8844-70d00948fb77" containerName="extract"
Jul 17 17:29:16.733578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.733437    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="79604c15-25ef-4898-8844-70d00948fb77" containerName="extract"
Jul 17 17:29:16.733578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.733502    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="79604c15-25ef-4898-8844-70d00948fb77" containerName="extract"
Jul 17 17:29:16.735444 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.735427    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kpxzl"
Jul 17 17:29:16.738103 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.738077    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hmq4x\""
Jul 17 17:29:16.744892 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.744867    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kpxzl"]
Jul 17 17:29:16.851296 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.851252    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77rnm\" (UniqueName: \"kubernetes.io/projected/4610ac79-4608-47ba-bb94-1c43ac209e5d-kube-api-access-77rnm\") pod \"cert-manager-55b4d954d5-kpxzl\" (UID: \"4610ac79-4608-47ba-bb94-1c43ac209e5d\") " pod="cert-manager/cert-manager-55b4d954d5-kpxzl"
Jul 17 17:29:16.851467 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.851354    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4610ac79-4608-47ba-bb94-1c43ac209e5d-bound-sa-token\") pod \"cert-manager-55b4d954d5-kpxzl\" (UID: \"4610ac79-4608-47ba-bb94-1c43ac209e5d\") " pod="cert-manager/cert-manager-55b4d954d5-kpxzl"
Jul 17 17:29:16.942298 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.942258    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cckv8"
Jul 17 17:29:16.952437 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.952405    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4610ac79-4608-47ba-bb94-1c43ac209e5d-bound-sa-token\") pod \"cert-manager-55b4d954d5-kpxzl\" (UID: \"4610ac79-4608-47ba-bb94-1c43ac209e5d\") " pod="cert-manager/cert-manager-55b4d954d5-kpxzl"
Jul 17 17:29:16.952604 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.952461    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77rnm\" (UniqueName: \"kubernetes.io/projected/4610ac79-4608-47ba-bb94-1c43ac209e5d-kube-api-access-77rnm\") pod \"cert-manager-55b4d954d5-kpxzl\" (UID: \"4610ac79-4608-47ba-bb94-1c43ac209e5d\") " pod="cert-manager/cert-manager-55b4d954d5-kpxzl"
Jul 17 17:29:16.962133 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.962104    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4610ac79-4608-47ba-bb94-1c43ac209e5d-bound-sa-token\") pod \"cert-manager-55b4d954d5-kpxzl\" (UID: \"4610ac79-4608-47ba-bb94-1c43ac209e5d\") " pod="cert-manager/cert-manager-55b4d954d5-kpxzl"
Jul 17 17:29:16.962313 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:16.962294    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77rnm\" (UniqueName: \"kubernetes.io/projected/4610ac79-4608-47ba-bb94-1c43ac209e5d-kube-api-access-77rnm\") pod \"cert-manager-55b4d954d5-kpxzl\" (UID: \"4610ac79-4608-47ba-bb94-1c43ac209e5d\") " pod="cert-manager/cert-manager-55b4d954d5-kpxzl"
Jul 17 17:29:17.045308 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:17.045202    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kpxzl"
Jul 17 17:29:17.185091 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:17.185061    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kpxzl"]
Jul 17 17:29:17.187780 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:29:17.187751    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4610ac79_4608_47ba_bb94_1c43ac209e5d.slice/crio-5e79b83b30a6c2f5953bd27e4fa38a63219d0b93fe4da752eb6a812c9e77f9cf WatchSource:0}: Error finding container 5e79b83b30a6c2f5953bd27e4fa38a63219d0b93fe4da752eb6a812c9e77f9cf: Status 404 returned error can't find the container with id 5e79b83b30a6c2f5953bd27e4fa38a63219d0b93fe4da752eb6a812c9e77f9cf
Jul 17 17:29:17.968660 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:17.968625    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kpxzl" event={"ID":"4610ac79-4608-47ba-bb94-1c43ac209e5d","Type":"ContainerStarted","Data":"d2277d439fc5e4cb997b7e13878ba92f797f2bfe69c6f416ab8677589efe6e0d"}
Jul 17 17:29:17.969040 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:17.968666    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kpxzl" event={"ID":"4610ac79-4608-47ba-bb94-1c43ac209e5d","Type":"ContainerStarted","Data":"5e79b83b30a6c2f5953bd27e4fa38a63219d0b93fe4da752eb6a812c9e77f9cf"}
Jul 17 17:29:17.983649 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:17.983591    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-kpxzl" podStartSLOduration=1.983573751 podStartE2EDuration="1.983573751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:29:17.982936574 +0000 UTC m=+329.988292429" watchObservedRunningTime="2026-07-17 17:29:17.983573751 +0000 UTC m=+329.988929604"
Jul 17 17:29:32.027544 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.027506    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"]
Jul 17 17:29:32.029810 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.029792    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.032624 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.032595    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 17 17:29:32.033862 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.033840    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 17 17:29:32.033985 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.033840    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pq2bx\""
Jul 17 17:29:32.039398 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.039356    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"]
Jul 17 17:29:32.061229 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.061181    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.061229 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.061236    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.061514 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.061287    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95bwm\" (UniqueName: \"kubernetes.io/projected/9a64aede-361e-4d39-9f49-20cdb9865fd5-kube-api-access-95bwm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.162343 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.162306    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.162583 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.162381    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95bwm\" (UniqueName: \"kubernetes.io/projected/9a64aede-361e-4d39-9f49-20cdb9865fd5-kube-api-access-95bwm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.162583 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.162450    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.162780 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.162755    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.162862 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.162817    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.172674 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.172645    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95bwm\" (UniqueName: \"kubernetes.io/projected/9a64aede-361e-4d39-9f49-20cdb9865fd5-kube-api-access-95bwm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.340637 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.340600    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:32.475997 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:32.475959    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"]
Jul 17 17:29:32.480105 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:29:32.480074    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a64aede_361e_4d39_9f49_20cdb9865fd5.slice/crio-daee038a3a58677e3978ddc80b22903e171a36c182bd4c25c741ea1b0f8632f2 WatchSource:0}: Error finding container daee038a3a58677e3978ddc80b22903e171a36c182bd4c25c741ea1b0f8632f2: Status 404 returned error can't find the container with id daee038a3a58677e3978ddc80b22903e171a36c182bd4c25c741ea1b0f8632f2
Jul 17 17:29:33.032039 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:33.032001    2572 generic.go:356] "Generic (PLEG): container finished" podID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerID="4ed9122c3a87f426996b86e4d892c48ee7b191628fd2a332254c039a2d9ae704" exitCode=0
Jul 17 17:29:33.032529 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:33.032090    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c" event={"ID":"9a64aede-361e-4d39-9f49-20cdb9865fd5","Type":"ContainerDied","Data":"4ed9122c3a87f426996b86e4d892c48ee7b191628fd2a332254c039a2d9ae704"}
Jul 17 17:29:33.032529 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:33.032143    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c" event={"ID":"9a64aede-361e-4d39-9f49-20cdb9865fd5","Type":"ContainerStarted","Data":"daee038a3a58677e3978ddc80b22903e171a36c182bd4c25c741ea1b0f8632f2"}
Jul 17 17:29:34.037988 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:34.037952    2572 generic.go:356] "Generic (PLEG): container finished" podID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerID="74cb53e906f4d8e35eb9cabd74ee2c6a8335fb6bbfb77f255583b8b9a965059b" exitCode=0
Jul 17 17:29:34.038389 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:34.038041    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c" event={"ID":"9a64aede-361e-4d39-9f49-20cdb9865fd5","Type":"ContainerDied","Data":"74cb53e906f4d8e35eb9cabd74ee2c6a8335fb6bbfb77f255583b8b9a965059b"}
Jul 17 17:29:35.044436 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:35.044393    2572 generic.go:356] "Generic (PLEG): container finished" podID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerID="e81c79b218fb3f1fb4bc8b317fe705fe1a4038c84167d610600ac8c9775494fa" exitCode=0
Jul 17 17:29:35.044843 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:35.044453    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c" event={"ID":"9a64aede-361e-4d39-9f49-20cdb9865fd5","Type":"ContainerDied","Data":"e81c79b218fb3f1fb4bc8b317fe705fe1a4038c84167d610600ac8c9775494fa"}
Jul 17 17:29:36.181393 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.181364    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:36.192776 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.192738    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95bwm\" (UniqueName: \"kubernetes.io/projected/9a64aede-361e-4d39-9f49-20cdb9865fd5-kube-api-access-95bwm\") pod \"9a64aede-361e-4d39-9f49-20cdb9865fd5\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") "
Jul 17 17:29:36.192933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.192791    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-bundle\") pod \"9a64aede-361e-4d39-9f49-20cdb9865fd5\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") "
Jul 17 17:29:36.192933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.192836    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-util\") pod \"9a64aede-361e-4d39-9f49-20cdb9865fd5\" (UID: \"9a64aede-361e-4d39-9f49-20cdb9865fd5\") "
Jul 17 17:29:36.193512 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.193461    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-bundle" (OuterVolumeSpecName: "bundle") pod "9a64aede-361e-4d39-9f49-20cdb9865fd5" (UID: "9a64aede-361e-4d39-9f49-20cdb9865fd5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:29:36.194996 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.194968    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a64aede-361e-4d39-9f49-20cdb9865fd5-kube-api-access-95bwm" (OuterVolumeSpecName: "kube-api-access-95bwm") pod "9a64aede-361e-4d39-9f49-20cdb9865fd5" (UID: "9a64aede-361e-4d39-9f49-20cdb9865fd5"). InnerVolumeSpecName "kube-api-access-95bwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:29:36.198730 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.198691    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-util" (OuterVolumeSpecName: "util") pod "9a64aede-361e-4d39-9f49-20cdb9865fd5" (UID: "9a64aede-361e-4d39-9f49-20cdb9865fd5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:29:36.293438 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.293394    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-util\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:29:36.293438 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.293427    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95bwm\" (UniqueName: \"kubernetes.io/projected/9a64aede-361e-4d39-9f49-20cdb9865fd5-kube-api-access-95bwm\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:29:36.293438 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:36.293440    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a64aede-361e-4d39-9f49-20cdb9865fd5-bundle\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:29:37.055581 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:37.055548    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c" event={"ID":"9a64aede-361e-4d39-9f49-20cdb9865fd5","Type":"ContainerDied","Data":"daee038a3a58677e3978ddc80b22903e171a36c182bd4c25c741ea1b0f8632f2"}
Jul 17 17:29:37.055751 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:37.055600    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="daee038a3a58677e3978ddc80b22903e171a36c182bd4c25c741ea1b0f8632f2"
Jul 17 17:29:37.055751 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:37.055558    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkcd8c"
Jul 17 17:29:45.401053 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.400968    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"]
Jul 17 17:29:45.401533 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.401340    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerName="util"
Jul 17 17:29:45.401533 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.401353    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerName="util"
Jul 17 17:29:45.401533 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.401366    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerName="extract"
Jul 17 17:29:45.401533 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.401371    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerName="extract"
Jul 17 17:29:45.401533 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.401383    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerName="pull"
Jul 17 17:29:45.401533 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.401388    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerName="pull"
Jul 17 17:29:45.401533 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.401440    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a64aede-361e-4d39-9f49-20cdb9865fd5" containerName="extract"
Jul 17 17:29:45.404580 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.404550    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.407183 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.407153    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 17:29:45.407338 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.407161    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l7hd7\""
Jul 17 17:29:45.407338 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.407163    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 17:29:45.407454 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.407341    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 17:29:45.407671 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.407655    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 17:29:45.426438 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.426397    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"]
Jul 17 17:29:45.465272 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.465226    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2f543b5-1949-4748-b631-bf9a52ab9052-tmp\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.465272 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.465273    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4pbd\" (UniqueName: \"kubernetes.io/projected/e2f543b5-1949-4748-b631-bf9a52ab9052-kube-api-access-k4pbd\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.465574 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.465294    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e2f543b5-1949-4748-b631-bf9a52ab9052-manifests\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.465574 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.465330    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e2f543b5-1949-4748-b631-bf9a52ab9052-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.465574 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.465409    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e2f543b5-1949-4748-b631-bf9a52ab9052-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.566461 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.566410    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e2f543b5-1949-4748-b631-bf9a52ab9052-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.566691 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.566474    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e2f543b5-1949-4748-b631-bf9a52ab9052-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.566691 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.566536    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2f543b5-1949-4748-b631-bf9a52ab9052-tmp\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.566691 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.566564    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pbd\" (UniqueName: \"kubernetes.io/projected/e2f543b5-1949-4748-b631-bf9a52ab9052-kube-api-access-k4pbd\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.566691 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.566590    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e2f543b5-1949-4748-b631-bf9a52ab9052-manifests\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.567047 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.567018    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e2f543b5-1949-4748-b631-bf9a52ab9052-manifests\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.568908 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.568872    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2f543b5-1949-4748-b631-bf9a52ab9052-tmp\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.569087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.569067    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e2f543b5-1949-4748-b631-bf9a52ab9052-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.569176 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.569157    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e2f543b5-1949-4748-b631-bf9a52ab9052-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.575058 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.575034    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4pbd\" (UniqueName: \"kubernetes.io/projected/e2f543b5-1949-4748-b631-bf9a52ab9052-kube-api-access-k4pbd\") pod \"opendatahub-operator-controller-manager-6cbb795d5f-vsc6t\" (UID: \"e2f543b5-1949-4748-b631-bf9a52ab9052\") " pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.715719 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.715622    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:45.877587 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:45.877559    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"]
Jul 17 17:29:45.880250 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:29:45.880217    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f543b5_1949_4748_b631_bf9a52ab9052.slice/crio-7091f5e5be33b3d8b5c05f74657e8ac2d77457d81921bfd9d330c3765410629b WatchSource:0}: Error finding container 7091f5e5be33b3d8b5c05f74657e8ac2d77457d81921bfd9d330c3765410629b: Status 404 returned error can't find the container with id 7091f5e5be33b3d8b5c05f74657e8ac2d77457d81921bfd9d330c3765410629b
Jul 17 17:29:46.097502 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:46.097451    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t" event={"ID":"e2f543b5-1949-4748-b631-bf9a52ab9052","Type":"ContainerStarted","Data":"7091f5e5be33b3d8b5c05f74657e8ac2d77457d81921bfd9d330c3765410629b"}
Jul 17 17:29:49.115794 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:49.115681    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t" event={"ID":"e2f543b5-1949-4748-b631-bf9a52ab9052","Type":"ContainerStarted","Data":"582f964c24af11c073f0c73a56b4afd6c8ddfa77b8b7a51575ab35c3e7175eef"}
Jul 17 17:29:50.121747 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:50.121702    2572 generic.go:356] "Generic (PLEG): container finished" podID="e2f543b5-1949-4748-b631-bf9a52ab9052" containerID="582f964c24af11c073f0c73a56b4afd6c8ddfa77b8b7a51575ab35c3e7175eef" exitCode=0
Jul 17 17:29:50.122149 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:50.121794    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t" event={"ID":"e2f543b5-1949-4748-b631-bf9a52ab9052","Type":"ContainerDied","Data":"582f964c24af11c073f0c73a56b4afd6c8ddfa77b8b7a51575ab35c3e7175eef"}
Jul 17 17:29:51.128587 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:51.128550    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t" event={"ID":"e2f543b5-1949-4748-b631-bf9a52ab9052","Type":"ContainerStarted","Data":"13d6868e14e5c387ffecbd573204e1b93681c16a92cf732cc42a14b986fe00f6"}
Jul 17 17:29:51.128991 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:51.128668    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:29:51.150509 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:29:51.150425    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t" podStartSLOduration=1.8189751360000002 podStartE2EDuration="6.150404736s" totalImagesPullingTime="3.275578521s" totalInitContainerRuntime="1.055851079s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:29:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:29:51.148756989 +0000 UTC m=+363.154112861" watchObservedRunningTime="2026-07-17 17:29:51.150404736 +0000 UTC m=+363.155760590"
Jul 17 17:30:02.135585 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:02.135554    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cbb795d5f-vsc6t"
Jul 17 17:30:03.307585 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.307529    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"]
Jul 17 17:30:03.316113 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.316087    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.320411 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.320384    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 17 17:30:03.320679 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.320384    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 17 17:30:03.320814 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.320384    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-swl4t\""
Jul 17 17:30:03.320914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.320526    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 17 17:30:03.321071 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.320581    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 17 17:30:03.321142 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.320584    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 17 17:30:03.322900 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.322874    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"]
Jul 17 17:30:03.414422 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.414373    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q9b6\" (UniqueName: \"kubernetes.io/projected/817a2fe7-7397-4e72-b89a-3cd732c2dc43-kube-api-access-2q9b6\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.414656 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.414453    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/817a2fe7-7397-4e72-b89a-3cd732c2dc43-manager-config\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.414656 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.414483    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/817a2fe7-7397-4e72-b89a-3cd732c2dc43-cert\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.414656 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.414532    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/817a2fe7-7397-4e72-b89a-3cd732c2dc43-metrics-cert\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.515790 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.515736    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/817a2fe7-7397-4e72-b89a-3cd732c2dc43-manager-config\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.515790 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.515797    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/817a2fe7-7397-4e72-b89a-3cd732c2dc43-cert\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.516066 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.515829    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/817a2fe7-7397-4e72-b89a-3cd732c2dc43-metrics-cert\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.516066 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.515896    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q9b6\" (UniqueName: \"kubernetes.io/projected/817a2fe7-7397-4e72-b89a-3cd732c2dc43-kube-api-access-2q9b6\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.516547 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.516523    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/817a2fe7-7397-4e72-b89a-3cd732c2dc43-manager-config\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.518279 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.518248    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/817a2fe7-7397-4e72-b89a-3cd732c2dc43-cert\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.518279 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.518277    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/817a2fe7-7397-4e72-b89a-3cd732c2dc43-metrics-cert\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.524642 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.524613    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q9b6\" (UniqueName: \"kubernetes.io/projected/817a2fe7-7397-4e72-b89a-3cd732c2dc43-kube-api-access-2q9b6\") pod \"lws-controller-manager-6d77fc5c7f-9srws\" (UID: \"817a2fe7-7397-4e72-b89a-3cd732c2dc43\") " pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.627895 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.627784    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:03.781906 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:03.781866    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"]
Jul 17 17:30:04.187999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:04.187947    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws" event={"ID":"817a2fe7-7397-4e72-b89a-3cd732c2dc43","Type":"ContainerStarted","Data":"8d09c1523ea08d575709467e2e19bb83913395ebfa181eed6985fd51749620ad"}
Jul 17 17:30:10.217694 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:10.217647    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws" event={"ID":"817a2fe7-7397-4e72-b89a-3cd732c2dc43","Type":"ContainerStarted","Data":"c0c9f5c0dc81be92f8098a68b2b8bb548e0701c569e506d7f51db75a30b79693"}
Jul 17 17:30:10.218089 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:10.217748    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:10.253400 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:10.253343    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws" podStartSLOduration=1.894503721 podStartE2EDuration="7.25332604s" totalImagesPullingTime="5.358822319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:30:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:30:10.251657669 +0000 UTC m=+382.257013544" watchObservedRunningTime="2026-07-17 17:30:10.25332604 +0000 UTC m=+382.258681893"
Jul 17 17:30:19.055967 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.055926    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"]
Jul 17 17:30:19.059186 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.059168    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.065059 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.065016    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 17 17:30:19.065235 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.065172    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 17 17:30:19.065332 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.065313    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2m6c9\""
Jul 17 17:30:19.065789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.065774    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 17:30:19.066375 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.066357    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 17:30:19.066503 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.066384    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 17:30:19.066503 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.066477    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 17 17:30:19.109442 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.109407    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"]
Jul 17 17:30:19.155130 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.155091    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.155130 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.155128    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hczb\" (UniqueName: \"kubernetes.io/projected/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-kube-api-access-5hczb\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.155381 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.155155    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.155381 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.155233    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.155381 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.155299    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.155381 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.155337    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.155381 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.155367    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.256542 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.256479    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.256542 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.256544    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hczb\" (UniqueName: \"kubernetes.io/projected/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-kube-api-access-5hczb\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.256808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.256585    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.256808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.256627    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.256808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.256665    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.256808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.256697    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.256808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.256732    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.257301 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.257266    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.259358 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.259333    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.259457 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.259423    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.259457 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.259443    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.259555 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.259517    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.266186 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.266162    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.266289 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.266168    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hczb\" (UniqueName: \"kubernetes.io/projected/3cc37fd2-4752-410e-b411-9ff7aa84a0f0-kube-api-access-5hczb\") pod \"istiod-openshift-gateway-94bb8fbfd-xh54n\" (UID: \"3cc37fd2-4752-410e-b411-9ff7aa84a0f0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.369296 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.369198    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:19.574999 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:19.574965    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"]
Jul 17 17:30:19.577057 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:30:19.577023    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc37fd2_4752_410e_b411_9ff7aa84a0f0.slice/crio-e6b79462b202801b783e51a612fa6d2e5d67fd321f88e902f58115cf0e333d1f WatchSource:0}: Error finding container e6b79462b202801b783e51a612fa6d2e5d67fd321f88e902f58115cf0e333d1f: Status 404 returned error can't find the container with id e6b79462b202801b783e51a612fa6d2e5d67fd321f88e902f58115cf0e333d1f
Jul 17 17:30:20.263856 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:20.263812    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n" event={"ID":"3cc37fd2-4752-410e-b411-9ff7aa84a0f0","Type":"ContainerStarted","Data":"e6b79462b202801b783e51a612fa6d2e5d67fd321f88e902f58115cf0e333d1f"}
Jul 17 17:30:21.225085 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:21.225050    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d77fc5c7f-9srws"
Jul 17 17:30:22.076187 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:22.076147    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 17 17:30:22.076473 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:22.076221    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 17 17:30:22.283227 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:22.283185    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n" event={"ID":"3cc37fd2-4752-410e-b411-9ff7aa84a0f0","Type":"ContainerStarted","Data":"ed4999612a1b413979d94af9259f2830f3031f9a45cd91237eb5f36bf8d0e8fe"}
Jul 17 17:30:22.283418 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:22.283299    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:30:22.313634 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:22.313562    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n" podStartSLOduration=0.81691692 podStartE2EDuration="3.313536849s" totalImagesPullingTime="2.496619929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:30:22.310850058 +0000 UTC m=+394.316205912" watchObservedRunningTime="2026-07-17 17:30:22.313536849 +0000 UTC m=+394.318892705"
Jul 17 17:30:23.289395 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:30:23.289367    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xh54n"
Jul 17 17:31:17.686447 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.686358    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"]
Jul 17 17:31:17.689700 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.689677    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.694481 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.694447    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4njl9\""
Jul 17 17:31:17.695392 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.695364    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 17:31:17.695542 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.695441    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 17:31:17.703353 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.703319    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"]
Jul 17 17:31:17.863468 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.863426    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.863683 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.863517    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2hlt\" (UniqueName: \"kubernetes.io/projected/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-kube-api-access-q2hlt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.863683 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.863543    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.963998 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.963904    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.963998 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.963979    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2hlt\" (UniqueName: \"kubernetes.io/projected/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-kube-api-access-q2hlt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.964187 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.964008    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.964398 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.964374    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.964439 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.964382    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.974068 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.974031    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2hlt\" (UniqueName: \"kubernetes.io/projected/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-kube-api-access-q2hlt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:17.999801 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:17.999757    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:18.147397 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.147161    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"]
Jul 17 17:31:18.150456 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:31:18.150410    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec935f99_40e0_4d29_84ce_ec44cde8f7e3.slice/crio-ba4d69ec7e834215bbaedadac7074cacb9a2f4f4e70635f1d0005ccd16a30b3f WatchSource:0}: Error finding container ba4d69ec7e834215bbaedadac7074cacb9a2f4f4e70635f1d0005ccd16a30b3f: Status 404 returned error can't find the container with id ba4d69ec7e834215bbaedadac7074cacb9a2f4f4e70635f1d0005ccd16a30b3f
Jul 17 17:31:18.313031 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.312994    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"]
Jul 17 17:31:18.316364 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.316345    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.330425 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.330392    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"]
Jul 17 17:31:18.467738 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.467631    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbm78\" (UniqueName: \"kubernetes.io/projected/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-kube-api-access-dbm78\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.467738 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.467681    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.467971 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.467812    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.538828 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.538792    2572 generic.go:356] "Generic (PLEG): container finished" podID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerID="b41210de2d4cd96fc4cf13cdae74af89abbcdc73ce32f145139b9922ce3e398a" exitCode=0
Jul 17 17:31:18.539028 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.538879    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k" event={"ID":"ec935f99-40e0-4d29-84ce-ec44cde8f7e3","Type":"ContainerDied","Data":"b41210de2d4cd96fc4cf13cdae74af89abbcdc73ce32f145139b9922ce3e398a"}
Jul 17 17:31:18.539028 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.538929    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k" event={"ID":"ec935f99-40e0-4d29-84ce-ec44cde8f7e3","Type":"ContainerStarted","Data":"ba4d69ec7e834215bbaedadac7074cacb9a2f4f4e70635f1d0005ccd16a30b3f"}
Jul 17 17:31:18.569315 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.569279    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.569517 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.569345    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbm78\" (UniqueName: \"kubernetes.io/projected/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-kube-api-access-dbm78\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.569517 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.569370    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.569791 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.569774    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.569832 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.569775    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.583872 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.583838    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbm78\" (UniqueName: \"kubernetes.io/projected/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-kube-api-access-dbm78\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.654591 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.654551    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:18.808329 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.808296    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"]
Jul 17 17:31:18.810249 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:31:18.810221    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941a052d_1fda_4c9c_9a51_3d2a6d6c5004.slice/crio-77f6e8ccc41d4802d70a825085c8ab9465f7902824813b1d81ccfe0c692d904c WatchSource:0}: Error finding container 77f6e8ccc41d4802d70a825085c8ab9465f7902824813b1d81ccfe0c692d904c: Status 404 returned error can't find the container with id 77f6e8ccc41d4802d70a825085c8ab9465f7902824813b1d81ccfe0c692d904c
Jul 17 17:31:18.906899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.906848    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"]
Jul 17 17:31:18.910171 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.910149    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:18.924284 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:18.924253    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"]
Jul 17 17:31:19.074794 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.074757    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.074794 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.074800    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjblk\" (UniqueName: \"kubernetes.io/projected/38e054be-1601-4e0e-ad96-a6748081b88d-kube-api-access-xjblk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.075033 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.074899    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.175996 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.175949    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.175996 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.175998    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjblk\" (UniqueName: \"kubernetes.io/projected/38e054be-1601-4e0e-ad96-a6748081b88d-kube-api-access-xjblk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.176222 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.176041    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.176392 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.176373    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.176435 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.176390    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.186015 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.185973    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjblk\" (UniqueName: \"kubernetes.io/projected/38e054be-1601-4e0e-ad96-a6748081b88d-kube-api-access-xjblk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.257415 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.257371    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:19.472870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.472834    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"]
Jul 17 17:31:19.479611 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.479585    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.490016 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.488382    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"]
Jul 17 17:31:19.523428 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.523390    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"]
Jul 17 17:31:19.526821 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:31:19.526780    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e054be_1601_4e0e_ad96_a6748081b88d.slice/crio-a182f2d770cf54a4049b6790ad5fa69df9963aabc78b986587454d6f158a83d1 WatchSource:0}: Error finding container a182f2d770cf54a4049b6790ad5fa69df9963aabc78b986587454d6f158a83d1: Status 404 returned error can't find the container with id a182f2d770cf54a4049b6790ad5fa69df9963aabc78b986587454d6f158a83d1
Jul 17 17:31:19.546939 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.546904    2572 generic.go:356] "Generic (PLEG): container finished" podID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerID="8b650def74100e63bee83ee6d3243a8c31e4e5242638d3c4576770ca145d739c" exitCode=0
Jul 17 17:31:19.547056 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.546947    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k" event={"ID":"ec935f99-40e0-4d29-84ce-ec44cde8f7e3","Type":"ContainerDied","Data":"8b650def74100e63bee83ee6d3243a8c31e4e5242638d3c4576770ca145d739c"}
Jul 17 17:31:19.548275 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.548242    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq" event={"ID":"38e054be-1601-4e0e-ad96-a6748081b88d","Type":"ContainerStarted","Data":"a182f2d770cf54a4049b6790ad5fa69df9963aabc78b986587454d6f158a83d1"}
Jul 17 17:31:19.549828 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.549800    2572 generic.go:356] "Generic (PLEG): container finished" podID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerID="303eb5157fa49020e1b83384b22df3802bc52643bd495ebe1aa8084ba7db883b" exitCode=0
Jul 17 17:31:19.549918 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.549875    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8" event={"ID":"941a052d-1fda-4c9c-9a51-3d2a6d6c5004","Type":"ContainerDied","Data":"303eb5157fa49020e1b83384b22df3802bc52643bd495ebe1aa8084ba7db883b"}
Jul 17 17:31:19.549956 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.549920    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8" event={"ID":"941a052d-1fda-4c9c-9a51-3d2a6d6c5004","Type":"ContainerStarted","Data":"77f6e8ccc41d4802d70a825085c8ab9465f7902824813b1d81ccfe0c692d904c"}
Jul 17 17:31:19.579613 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.579579    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.579795 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.579642    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.579795 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.579722    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tflnb\" (UniqueName: \"kubernetes.io/projected/47670189-5913-42dd-a2b9-aacef3d884af-kube-api-access-tflnb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.680296 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.680257    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tflnb\" (UniqueName: \"kubernetes.io/projected/47670189-5913-42dd-a2b9-aacef3d884af-kube-api-access-tflnb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.680486 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.680353    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.680486 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.680379    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.680862 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.680835    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.680862 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.680852    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.689613 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.689582    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tflnb\" (UniqueName: \"kubernetes.io/projected/47670189-5913-42dd-a2b9-aacef3d884af-kube-api-access-tflnb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.793367 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.793327    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:19.938361 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:19.938259    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"]
Jul 17 17:31:19.941134 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:31:19.941104    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47670189_5913_42dd_a2b9_aacef3d884af.slice/crio-170fc56f50b3885a89e2d38e361ea246f17a2028ba0076ec141cb493ea2113b9 WatchSource:0}: Error finding container 170fc56f50b3885a89e2d38e361ea246f17a2028ba0076ec141cb493ea2113b9: Status 404 returned error can't find the container with id 170fc56f50b3885a89e2d38e361ea246f17a2028ba0076ec141cb493ea2113b9
Jul 17 17:31:20.557539 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:20.557505    2572 generic.go:356] "Generic (PLEG): container finished" podID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerID="6d05c1fc739d85ad714c0283e098d9036b44568d29add681a23268adea23584d" exitCode=0
Jul 17 17:31:20.557733 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:20.557584    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k" event={"ID":"ec935f99-40e0-4d29-84ce-ec44cde8f7e3","Type":"ContainerDied","Data":"6d05c1fc739d85ad714c0283e098d9036b44568d29add681a23268adea23584d"}
Jul 17 17:31:20.559165 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:20.559133    2572 generic.go:356] "Generic (PLEG): container finished" podID="38e054be-1601-4e0e-ad96-a6748081b88d" containerID="0c8315bc697bbc30543b2f5c8922b20f033b06806be135edf78cfe07c9f68f3b" exitCode=0
Jul 17 17:31:20.559292 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:20.559211    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq" event={"ID":"38e054be-1601-4e0e-ad96-a6748081b88d","Type":"ContainerDied","Data":"0c8315bc697bbc30543b2f5c8922b20f033b06806be135edf78cfe07c9f68f3b"}
Jul 17 17:31:20.561224 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:20.561202    2572 generic.go:356] "Generic (PLEG): container finished" podID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerID="c1ad218c114a452f415ec1bb39a9f4ab9c27783b55337ee3633b09f7798ae55f" exitCode=0
Jul 17 17:31:20.561350 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:20.561293    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8" event={"ID":"941a052d-1fda-4c9c-9a51-3d2a6d6c5004","Type":"ContainerDied","Data":"c1ad218c114a452f415ec1bb39a9f4ab9c27783b55337ee3633b09f7798ae55f"}
Jul 17 17:31:20.563037 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:20.563013    2572 generic.go:356] "Generic (PLEG): container finished" podID="47670189-5913-42dd-a2b9-aacef3d884af" containerID="ae9f7069d2785671dd5fff8416aee1f2846c686b448aa4390a3f6cfed1c22bbb" exitCode=0
Jul 17 17:31:20.563120 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:20.563062    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h" event={"ID":"47670189-5913-42dd-a2b9-aacef3d884af","Type":"ContainerDied","Data":"ae9f7069d2785671dd5fff8416aee1f2846c686b448aa4390a3f6cfed1c22bbb"}
Jul 17 17:31:20.563120 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:20.563108    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h" event={"ID":"47670189-5913-42dd-a2b9-aacef3d884af","Type":"ContainerStarted","Data":"170fc56f50b3885a89e2d38e361ea246f17a2028ba0076ec141cb493ea2113b9"}
Jul 17 17:31:21.570639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.570551    2572 generic.go:356] "Generic (PLEG): container finished" podID="47670189-5913-42dd-a2b9-aacef3d884af" containerID="d37ba98d70e197a04522471f5776ba24b28118167cb27814f3e0c16a5bdcf447" exitCode=0
Jul 17 17:31:21.571146 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.570636    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h" event={"ID":"47670189-5913-42dd-a2b9-aacef3d884af","Type":"ContainerDied","Data":"d37ba98d70e197a04522471f5776ba24b28118167cb27814f3e0c16a5bdcf447"}
Jul 17 17:31:21.573273 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.573244    2572 generic.go:356] "Generic (PLEG): container finished" podID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerID="7207b2f24b3a53250bafba71c4774f2ba2592fe91f231251bca225972f975b76" exitCode=0
Jul 17 17:31:21.573411 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.573326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8" event={"ID":"941a052d-1fda-4c9c-9a51-3d2a6d6c5004","Type":"ContainerDied","Data":"7207b2f24b3a53250bafba71c4774f2ba2592fe91f231251bca225972f975b76"}
Jul 17 17:31:21.727263 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.727235    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:21.900052 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.899997    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-bundle\") pod \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") "
Jul 17 17:31:21.900265 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.900101    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-util\") pod \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") "
Jul 17 17:31:21.900265 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.900142    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2hlt\" (UniqueName: \"kubernetes.io/projected/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-kube-api-access-q2hlt\") pod \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\" (UID: \"ec935f99-40e0-4d29-84ce-ec44cde8f7e3\") "
Jul 17 17:31:21.900602 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.900572    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-bundle" (OuterVolumeSpecName: "bundle") pod "ec935f99-40e0-4d29-84ce-ec44cde8f7e3" (UID: "ec935f99-40e0-4d29-84ce-ec44cde8f7e3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:31:21.902639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.902614    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-kube-api-access-q2hlt" (OuterVolumeSpecName: "kube-api-access-q2hlt") pod "ec935f99-40e0-4d29-84ce-ec44cde8f7e3" (UID: "ec935f99-40e0-4d29-84ce-ec44cde8f7e3"). InnerVolumeSpecName "kube-api-access-q2hlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:31:21.905907 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:21.905868    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-util" (OuterVolumeSpecName: "util") pod "ec935f99-40e0-4d29-84ce-ec44cde8f7e3" (UID: "ec935f99-40e0-4d29-84ce-ec44cde8f7e3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:31:22.001564 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.001522    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-bundle\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:22.001564 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.001554    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-util\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:22.001564 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.001565    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2hlt\" (UniqueName: \"kubernetes.io/projected/ec935f99-40e0-4d29-84ce-ec44cde8f7e3-kube-api-access-q2hlt\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:22.582074 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.582034    2572 generic.go:356] "Generic (PLEG): container finished" podID="47670189-5913-42dd-a2b9-aacef3d884af" containerID="12d31e9cac1b90378abbf9090da833c67c995cd8869d7b13c590a755b494eced" exitCode=0
Jul 17 17:31:22.582869 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.582120    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h" event={"ID":"47670189-5913-42dd-a2b9-aacef3d884af","Type":"ContainerDied","Data":"12d31e9cac1b90378abbf9090da833c67c995cd8869d7b13c590a755b494eced"}
Jul 17 17:31:22.584235 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.584205    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k" event={"ID":"ec935f99-40e0-4d29-84ce-ec44cde8f7e3","Type":"ContainerDied","Data":"ba4d69ec7e834215bbaedadac7074cacb9a2f4f4e70635f1d0005ccd16a30b3f"}
Jul 17 17:31:22.584357 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.584253    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba4d69ec7e834215bbaedadac7074cacb9a2f4f4e70635f1d0005ccd16a30b3f"
Jul 17 17:31:22.584357 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.584218    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k"
Jul 17 17:31:22.586912 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.586884    2572 generic.go:356] "Generic (PLEG): container finished" podID="38e054be-1601-4e0e-ad96-a6748081b88d" containerID="665cccb1314071175f8444259b39cc9c1cae7e27d2bcb5966e9b9416355ae073" exitCode=0
Jul 17 17:31:22.587039 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.586962    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq" event={"ID":"38e054be-1601-4e0e-ad96-a6748081b88d","Type":"ContainerDied","Data":"665cccb1314071175f8444259b39cc9c1cae7e27d2bcb5966e9b9416355ae073"}
Jul 17 17:31:22.738044 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.738016    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:22.909564 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.909527    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbm78\" (UniqueName: \"kubernetes.io/projected/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-kube-api-access-dbm78\") pod \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") "
Jul 17 17:31:22.909564 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.909574    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-bundle\") pod \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") "
Jul 17 17:31:22.909780 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.909610    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-util\") pod \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\" (UID: \"941a052d-1fda-4c9c-9a51-3d2a6d6c5004\") "
Jul 17 17:31:22.910254 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.910221    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-bundle" (OuterVolumeSpecName: "bundle") pod "941a052d-1fda-4c9c-9a51-3d2a6d6c5004" (UID: "941a052d-1fda-4c9c-9a51-3d2a6d6c5004"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:31:22.911882 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.911849    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-kube-api-access-dbm78" (OuterVolumeSpecName: "kube-api-access-dbm78") pod "941a052d-1fda-4c9c-9a51-3d2a6d6c5004" (UID: "941a052d-1fda-4c9c-9a51-3d2a6d6c5004"). InnerVolumeSpecName "kube-api-access-dbm78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:31:22.915007 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:22.914973    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-util" (OuterVolumeSpecName: "util") pod "941a052d-1fda-4c9c-9a51-3d2a6d6c5004" (UID: "941a052d-1fda-4c9c-9a51-3d2a6d6c5004"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:31:23.010703 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.010659    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbm78\" (UniqueName: \"kubernetes.io/projected/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-kube-api-access-dbm78\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:23.010703 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.010696    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-bundle\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:23.010703 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.010707    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941a052d-1fda-4c9c-9a51-3d2a6d6c5004-util\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:23.593753 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.593718    2572 generic.go:356] "Generic (PLEG): container finished" podID="38e054be-1601-4e0e-ad96-a6748081b88d" containerID="5778f8cb28f32ab6a07c23290c2169ad197dfab5738ff62f6bf1d1c940522870" exitCode=0
Jul 17 17:31:23.594237 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.593811    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq" event={"ID":"38e054be-1601-4e0e-ad96-a6748081b88d","Type":"ContainerDied","Data":"5778f8cb28f32ab6a07c23290c2169ad197dfab5738ff62f6bf1d1c940522870"}
Jul 17 17:31:23.595820 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.595797    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8"
Jul 17 17:31:23.595820 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.595808    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8" event={"ID":"941a052d-1fda-4c9c-9a51-3d2a6d6c5004","Type":"ContainerDied","Data":"77f6e8ccc41d4802d70a825085c8ab9465f7902824813b1d81ccfe0c692d904c"}
Jul 17 17:31:23.595997 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.595853    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77f6e8ccc41d4802d70a825085c8ab9465f7902824813b1d81ccfe0c692d904c"
Jul 17 17:31:23.736849 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.736821    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:23.917791 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.917692    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-bundle\") pod \"47670189-5913-42dd-a2b9-aacef3d884af\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") "
Jul 17 17:31:23.917791 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.917761    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-util\") pod \"47670189-5913-42dd-a2b9-aacef3d884af\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") "
Jul 17 17:31:23.918000 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.917822    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tflnb\" (UniqueName: \"kubernetes.io/projected/47670189-5913-42dd-a2b9-aacef3d884af-kube-api-access-tflnb\") pod \"47670189-5913-42dd-a2b9-aacef3d884af\" (UID: \"47670189-5913-42dd-a2b9-aacef3d884af\") "
Jul 17 17:31:23.918231 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.918205    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-bundle" (OuterVolumeSpecName: "bundle") pod "47670189-5913-42dd-a2b9-aacef3d884af" (UID: "47670189-5913-42dd-a2b9-aacef3d884af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:31:23.920129 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.920102    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47670189-5913-42dd-a2b9-aacef3d884af-kube-api-access-tflnb" (OuterVolumeSpecName: "kube-api-access-tflnb") pod "47670189-5913-42dd-a2b9-aacef3d884af" (UID: "47670189-5913-42dd-a2b9-aacef3d884af"). InnerVolumeSpecName "kube-api-access-tflnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:31:23.923576 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:23.923533    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-util" (OuterVolumeSpecName: "util") pod "47670189-5913-42dd-a2b9-aacef3d884af" (UID: "47670189-5913-42dd-a2b9-aacef3d884af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:31:24.018810 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.018765    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tflnb\" (UniqueName: \"kubernetes.io/projected/47670189-5913-42dd-a2b9-aacef3d884af-kube-api-access-tflnb\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:24.018810 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.018800    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-bundle\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:24.018810 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.018810    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47670189-5913-42dd-a2b9-aacef3d884af-util\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:24.602552 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.602502    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h" event={"ID":"47670189-5913-42dd-a2b9-aacef3d884af","Type":"ContainerDied","Data":"170fc56f50b3885a89e2d38e361ea246f17a2028ba0076ec141cb493ea2113b9"}
Jul 17 17:31:24.602976 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.602569    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="170fc56f50b3885a89e2d38e361ea246f17a2028ba0076ec141cb493ea2113b9"
Jul 17 17:31:24.602976 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.602516    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h"
Jul 17 17:31:24.742696 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.742667    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:24.927799 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.927761    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjblk\" (UniqueName: \"kubernetes.io/projected/38e054be-1601-4e0e-ad96-a6748081b88d-kube-api-access-xjblk\") pod \"38e054be-1601-4e0e-ad96-a6748081b88d\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") "
Jul 17 17:31:24.927799 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.927806    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-util\") pod \"38e054be-1601-4e0e-ad96-a6748081b88d\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") "
Jul 17 17:31:24.928074 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.927837    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-bundle\") pod \"38e054be-1601-4e0e-ad96-a6748081b88d\" (UID: \"38e054be-1601-4e0e-ad96-a6748081b88d\") "
Jul 17 17:31:24.928407 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.928377    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-bundle" (OuterVolumeSpecName: "bundle") pod "38e054be-1601-4e0e-ad96-a6748081b88d" (UID: "38e054be-1601-4e0e-ad96-a6748081b88d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:31:24.930101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.930082    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38e054be-1601-4e0e-ad96-a6748081b88d-kube-api-access-xjblk" (OuterVolumeSpecName: "kube-api-access-xjblk") pod "38e054be-1601-4e0e-ad96-a6748081b88d" (UID: "38e054be-1601-4e0e-ad96-a6748081b88d"). InnerVolumeSpecName "kube-api-access-xjblk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:31:24.933216 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:24.933178    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-util" (OuterVolumeSpecName: "util") pod "38e054be-1601-4e0e-ad96-a6748081b88d" (UID: "38e054be-1601-4e0e-ad96-a6748081b88d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:31:25.029383 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:25.029334    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjblk\" (UniqueName: \"kubernetes.io/projected/38e054be-1601-4e0e-ad96-a6748081b88d-kube-api-access-xjblk\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:25.029383 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:25.029371    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-util\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:25.029383 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:25.029384    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38e054be-1601-4e0e-ad96-a6748081b88d-bundle\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:31:25.609638 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:25.609606    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq"
Jul 17 17:31:25.609638 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:25.609628    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq" event={"ID":"38e054be-1601-4e0e-ad96-a6748081b88d","Type":"ContainerDied","Data":"a182f2d770cf54a4049b6790ad5fa69df9963aabc78b986587454d6f158a83d1"}
Jul 17 17:31:25.610144 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:31:25.609683    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a182f2d770cf54a4049b6790ad5fa69df9963aabc78b986587454d6f158a83d1"
Jul 17 17:32:08.762883 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.762839    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"]
Jul 17 17:32:08.764146 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764116    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerName="extract"
Jul 17 17:32:08.764299 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764287    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerName="extract"
Jul 17 17:32:08.764379 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764369    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38e054be-1601-4e0e-ad96-a6748081b88d" containerName="pull"
Jul 17 17:32:08.764452 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764444    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="38e054be-1601-4e0e-ad96-a6748081b88d" containerName="pull"
Jul 17 17:32:08.764556 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764546    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerName="pull"
Jul 17 17:32:08.764627 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764619    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerName="pull"
Jul 17 17:32:08.764695 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764687    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38e054be-1601-4e0e-ad96-a6748081b88d" containerName="util"
Jul 17 17:32:08.764763 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764755    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="38e054be-1601-4e0e-ad96-a6748081b88d" containerName="util"
Jul 17 17:32:08.764846 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764835    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerName="util"
Jul 17 17:32:08.764912 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764904    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerName="util"
Jul 17 17:32:08.764986 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.764978    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerName="extract"
Jul 17 17:32:08.765050 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765041    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerName="extract"
Jul 17 17:32:08.765132 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765117    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47670189-5913-42dd-a2b9-aacef3d884af" containerName="pull"
Jul 17 17:32:08.765244 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765209    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="47670189-5913-42dd-a2b9-aacef3d884af" containerName="pull"
Jul 17 17:32:08.765375 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765290    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47670189-5913-42dd-a2b9-aacef3d884af" containerName="util"
Jul 17 17:32:08.765458 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765447    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="47670189-5913-42dd-a2b9-aacef3d884af" containerName="util"
Jul 17 17:32:08.765586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765459    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerName="pull"
Jul 17 17:32:08.765586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765465    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerName="pull"
Jul 17 17:32:08.765586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765475    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerName="util"
Jul 17 17:32:08.765586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765481    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerName="util"
Jul 17 17:32:08.765586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765513    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47670189-5913-42dd-a2b9-aacef3d884af" containerName="extract"
Jul 17 17:32:08.765586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765521    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="47670189-5913-42dd-a2b9-aacef3d884af" containerName="extract"
Jul 17 17:32:08.765586 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765535    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38e054be-1601-4e0e-ad96-a6748081b88d" containerName="extract"
Jul 17 17:32:08.765834 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765542    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="38e054be-1601-4e0e-ad96-a6748081b88d" containerName="extract"
Jul 17 17:32:08.765834 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765753    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="47670189-5913-42dd-a2b9-aacef3d884af" containerName="extract"
Jul 17 17:32:08.765834 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765768    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec935f99-40e0-4d29-84ce-ec44cde8f7e3" containerName="extract"
Jul 17 17:32:08.765834 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765775    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="38e054be-1601-4e0e-ad96-a6748081b88d" containerName="extract"
Jul 17 17:32:08.765834 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.765782    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="941a052d-1fda-4c9c-9a51-3d2a6d6c5004" containerName="extract"
Jul 17 17:32:08.770598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.770577    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:32:08.774057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.774040    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 17:32:08.774170 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.774042    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 17:32:08.774170 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.774042    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fkc44\""
Jul 17 17:32:08.781556 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.781527    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"]
Jul 17 17:32:08.913842 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.913800    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84s4v\" (UniqueName: \"kubernetes.io/projected/346eaea9-35bf-4d22-ac45-7074a0617994-kube-api-access-84s4v\") pod \"kuadrant-operator-controller-manager-86878c769f-vr4qx\" (UID: \"346eaea9-35bf-4d22-ac45-7074a0617994\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:32:08.913842 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:08.913849    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/346eaea9-35bf-4d22-ac45-7074a0617994-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vr4qx\" (UID: \"346eaea9-35bf-4d22-ac45-7074a0617994\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:32:09.014768 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:09.014676    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84s4v\" (UniqueName: \"kubernetes.io/projected/346eaea9-35bf-4d22-ac45-7074a0617994-kube-api-access-84s4v\") pod \"kuadrant-operator-controller-manager-86878c769f-vr4qx\" (UID: \"346eaea9-35bf-4d22-ac45-7074a0617994\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:32:09.014768 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:09.014719    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/346eaea9-35bf-4d22-ac45-7074a0617994-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vr4qx\" (UID: \"346eaea9-35bf-4d22-ac45-7074a0617994\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:32:09.015110 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:09.015068    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/346eaea9-35bf-4d22-ac45-7074a0617994-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vr4qx\" (UID: \"346eaea9-35bf-4d22-ac45-7074a0617994\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:32:09.025720 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:09.025688    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84s4v\" (UniqueName: \"kubernetes.io/projected/346eaea9-35bf-4d22-ac45-7074a0617994-kube-api-access-84s4v\") pod \"kuadrant-operator-controller-manager-86878c769f-vr4qx\" (UID: \"346eaea9-35bf-4d22-ac45-7074a0617994\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:32:09.081805 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:09.081767    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:32:09.247227 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:09.247188    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"]
Jul 17 17:32:09.252775 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:32:09.252745    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346eaea9_35bf_4d22_ac45_7074a0617994.slice/crio-455a6e4b84fd311182df3b16573ffde174410b981ce1867167d0b5bb14c03eb5 WatchSource:0}: Error finding container 455a6e4b84fd311182df3b16573ffde174410b981ce1867167d0b5bb14c03eb5: Status 404 returned error can't find the container with id 455a6e4b84fd311182df3b16573ffde174410b981ce1867167d0b5bb14c03eb5
Jul 17 17:32:09.826990 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:09.826951    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx" event={"ID":"346eaea9-35bf-4d22-ac45-7074a0617994","Type":"ContainerStarted","Data":"455a6e4b84fd311182df3b16573ffde174410b981ce1867167d0b5bb14c03eb5"}
Jul 17 17:32:14.854037 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:14.853990    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx" event={"ID":"346eaea9-35bf-4d22-ac45-7074a0617994","Type":"ContainerStarted","Data":"1ea01b827b7918127c6c7529a883e26bdaed87fd098657e8dd207008dacdce74"}
Jul 17 17:32:14.854574 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:14.854082    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:32:14.877604 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:14.877546    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx" podStartSLOduration=1.8008048140000001 podStartE2EDuration="6.877529028s" totalImagesPullingTime="5.076724214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:32:14.875769425 +0000 UTC m=+506.881125292" watchObservedRunningTime="2026-07-17 17:32:14.877529028 +0000 UTC m=+506.882884880"
Jul 17 17:32:25.861598 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:32:25.861565    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:33:48.456849 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:33:48.456817    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:33:48.460795 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:33:48.460762    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:33:48.464006 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:33:48.463977    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:33:48.467622 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:33:48.467597    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:34:03.195511 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.195449    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"]
Jul 17 17:34:03.197756 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.197740    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:03.202511 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:34:03.202439    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-dockercfg-68584\" is forbidden: User \"system:node:ip-10-0-135-30.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-68584\"" type="*v1.Secret"
Jul 17 17:34:03.202612 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:34:03.202559    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-pre-processing-7677b774bb-jdrnd\" is forbidden: User \"system:node:ip-10-0-135-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-30.ec2.internal' and this object" podUID="a3e83a56-c4d7-4a82-a983-48c17c880fb0" pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:03.202612 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:34:03.202588    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"payload-processing-plugins\" is forbidden: User \"system:node:ip-10-0-135-30.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" type="*v1.ConfigMap"
Jul 17 17:34:03.223368 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.223329    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"]
Jul 17 17:34:03.287209 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.287165    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-nn7kv"]
Jul 17 17:34:03.289566 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.289540    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:03.294161 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:34:03.294121    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-processing-5bf47cd988-nn7kv\" is forbidden: User \"system:node:ip-10-0-135-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-30.ec2.internal' and this object" podUID="e2e1d612-7d04-4e29-9813-fd61d6e0a63a" pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:03.310850 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.310793    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxx24\" (UniqueName: \"kubernetes.io/projected/a3e83a56-c4d7-4a82-a983-48c17c880fb0-kube-api-access-zxx24\") pod \"payload-pre-processing-7677b774bb-jdrnd\" (UID: \"a3e83a56-c4d7-4a82-a983-48c17c880fb0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:03.310850 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.310840    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e83a56-c4d7-4a82-a983-48c17c880fb0-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-jdrnd\" (UID: \"a3e83a56-c4d7-4a82-a983-48c17c880fb0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:03.314172 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.314139    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-nn7kv"]
Jul 17 17:34:03.411875 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.411841    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmltc\" (UniqueName: \"kubernetes.io/projected/e2e1d612-7d04-4e29-9813-fd61d6e0a63a-kube-api-access-qmltc\") pod \"payload-processing-5bf47cd988-nn7kv\" (UID: \"e2e1d612-7d04-4e29-9813-fd61d6e0a63a\") " pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:03.412073 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.411895    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxx24\" (UniqueName: \"kubernetes.io/projected/a3e83a56-c4d7-4a82-a983-48c17c880fb0-kube-api-access-zxx24\") pod \"payload-pre-processing-7677b774bb-jdrnd\" (UID: \"a3e83a56-c4d7-4a82-a983-48c17c880fb0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:03.412073 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.411953    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e83a56-c4d7-4a82-a983-48c17c880fb0-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-jdrnd\" (UID: \"a3e83a56-c4d7-4a82-a983-48c17c880fb0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:03.412073 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.411977    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2e1d612-7d04-4e29-9813-fd61d6e0a63a-plugins-config-volume\") pod \"payload-processing-5bf47cd988-nn7kv\" (UID: \"e2e1d612-7d04-4e29-9813-fd61d6e0a63a\") " pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:03.434361 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.434322    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxx24\" (UniqueName: \"kubernetes.io/projected/a3e83a56-c4d7-4a82-a983-48c17c880fb0-kube-api-access-zxx24\") pod \"payload-pre-processing-7677b774bb-jdrnd\" (UID: \"a3e83a56-c4d7-4a82-a983-48c17c880fb0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:03.512833 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.512745    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmltc\" (UniqueName: \"kubernetes.io/projected/e2e1d612-7d04-4e29-9813-fd61d6e0a63a-kube-api-access-qmltc\") pod \"payload-processing-5bf47cd988-nn7kv\" (UID: \"e2e1d612-7d04-4e29-9813-fd61d6e0a63a\") " pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:03.512833 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.512789    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2e1d612-7d04-4e29-9813-fd61d6e0a63a-plugins-config-volume\") pod \"payload-processing-5bf47cd988-nn7kv\" (UID: \"e2e1d612-7d04-4e29-9813-fd61d6e0a63a\") " pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:03.528354 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:03.528320    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmltc\" (UniqueName: \"kubernetes.io/projected/e2e1d612-7d04-4e29-9813-fd61d6e0a63a-kube-api-access-qmltc\") pod \"payload-processing-5bf47cd988-nn7kv\" (UID: \"e2e1d612-7d04-4e29-9813-fd61d6e0a63a\") " pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:04.131790 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:04.131740    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 17 17:34:04.133294 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:04.133264    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e83a56-c4d7-4a82-a983-48c17c880fb0-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-jdrnd\" (UID: \"a3e83a56-c4d7-4a82-a983-48c17c880fb0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:04.133825 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:04.133809    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2e1d612-7d04-4e29-9813-fd61d6e0a63a-plugins-config-volume\") pod \"payload-processing-5bf47cd988-nn7kv\" (UID: \"e2e1d612-7d04-4e29-9813-fd61d6e0a63a\") " pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:04.236286 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:04.236244    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-68584\""
Jul 17 17:34:04.241829 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:04.241797    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:04.401599 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:04.401570    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-nn7kv"]
Jul 17 17:34:04.404157 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:34:04.404125    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e1d612_7d04_4e29_9813_fd61d6e0a63a.slice/crio-05515b6230ea6c683905d748e776391d7b2f4ed60ee2ef079e220575924d968a WatchSource:0}: Error finding container 05515b6230ea6c683905d748e776391d7b2f4ed60ee2ef079e220575924d968a: Status 404 returned error can't find the container with id 05515b6230ea6c683905d748e776391d7b2f4ed60ee2ef079e220575924d968a
Jul 17 17:34:04.406438 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:04.406420    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 17:34:04.408046 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:04.408027    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:04.569223 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:04.569194    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"]
Jul 17 17:34:04.570727 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:34:04.570694    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e83a56_c4d7_4a82_a983_48c17c880fb0.slice/crio-178b7bfd8eaff1f8b4b75d8c4d5339efcd1ff51afb21937ee6297496252fe6f5 WatchSource:0}: Error finding container 178b7bfd8eaff1f8b4b75d8c4d5339efcd1ff51afb21937ee6297496252fe6f5: Status 404 returned error can't find the container with id 178b7bfd8eaff1f8b4b75d8c4d5339efcd1ff51afb21937ee6297496252fe6f5
Jul 17 17:34:05.401375 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:05.401319    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv" event={"ID":"e2e1d612-7d04-4e29-9813-fd61d6e0a63a","Type":"ContainerStarted","Data":"05515b6230ea6c683905d748e776391d7b2f4ed60ee2ef079e220575924d968a"}
Jul 17 17:34:05.403485 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:05.403452    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd" event={"ID":"a3e83a56-c4d7-4a82-a983-48c17c880fb0","Type":"ContainerStarted","Data":"178b7bfd8eaff1f8b4b75d8c4d5339efcd1ff51afb21937ee6297496252fe6f5"}
Jul 17 17:34:06.410180 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:06.410136    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv" event={"ID":"e2e1d612-7d04-4e29-9813-fd61d6e0a63a","Type":"ContainerStarted","Data":"9acba2b1b9db0405173daefe9496f7e9830fd24d31913e7b24a286280b6f317f"}
Jul 17 17:34:06.410742 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:06.410332    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:06.411831 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:06.411797    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd" event={"ID":"a3e83a56-c4d7-4a82-a983-48c17c880fb0","Type":"ContainerStarted","Data":"4dcfc4b42e7cd09987ff0e5eb7cb581b5047bd1b246fc134ac72ffc52b2aa9f3"}
Jul 17 17:34:06.411952 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:06.411855    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:06.432979 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:06.432918    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv" podStartSLOduration=1.633993093 podStartE2EDuration="3.432898002s" totalImagesPullingTime="1.798904909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:34:06.431712292 +0000 UTC m=+618.437068169" watchObservedRunningTime="2026-07-17 17:34:06.432898002 +0000 UTC m=+618.438253855"
Jul 17 17:34:06.470574 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:06.470404    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd" podStartSLOduration=1.84460336 podStartE2EDuration="3.470385954s" totalImagesPullingTime="1.625782594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:34:06.468261018 +0000 UTC m=+618.473616874" watchObservedRunningTime="2026-07-17 17:34:06.470385954 +0000 UTC m=+618.475741807"
Jul 17 17:34:17.421738 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:17.421652    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-nn7kv"
Jul 17 17:34:17.421738 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:17.421734    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-jdrnd"
Jul 17 17:34:25.384634 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.384598    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"]
Jul 17 17:34:25.386895 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.386880    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.389661 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.389632    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wsgb4\""
Jul 17 17:34:25.390716 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.390692    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 17 17:34:25.390826 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.390692    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 17:34:25.390826 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.390771    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 17:34:25.400714 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.400689    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"]
Jul 17 17:34:25.498917 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.498874    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.498917 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.498918    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.499187 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.498956    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.499187 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.499021    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzqvc\" (UniqueName: \"kubernetes.io/projected/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-kube-api-access-zzqvc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.499187 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.499073    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.499187 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.499104    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.599997 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.599932    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.599997 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.600006    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzqvc\" (UniqueName: \"kubernetes.io/projected/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-kube-api-access-zzqvc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.600269 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.600059    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.600269 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.600086    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.600269 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.600112    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.600269 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.600136    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.600535 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.600460    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.600610 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.600535    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.600610 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.600563    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.602338 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.602315    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.602816 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.602794    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.609605 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.609576    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzqvc\" (UniqueName: \"kubernetes.io/projected/aa88a0b9-d2ad-40ae-8dba-1dec3938d94c-kube-api-access-zzqvc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l\" (UID: \"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.697588 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.697465    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:25.844347 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:25.844316    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"]
Jul 17 17:34:25.846011 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:34:25.845984    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa88a0b9_d2ad_40ae_8dba_1dec3938d94c.slice/crio-390d4f9a14496b69595d239939e9e95aa6982e9dd54204d161471d93f035f2c5 WatchSource:0}: Error finding container 390d4f9a14496b69595d239939e9e95aa6982e9dd54204d161471d93f035f2c5: Status 404 returned error can't find the container with id 390d4f9a14496b69595d239939e9e95aa6982e9dd54204d161471d93f035f2c5
Jul 17 17:34:26.521925 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:26.521878    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l" event={"ID":"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c","Type":"ContainerStarted","Data":"390d4f9a14496b69595d239939e9e95aa6982e9dd54204d161471d93f035f2c5"}
Jul 17 17:34:27.528703 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:27.528663    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l" event={"ID":"aa88a0b9-d2ad-40ae-8dba-1dec3938d94c","Type":"ContainerStarted","Data":"1df0bf31fd125d99b01b8988d2476b989ec0da7331b8bcb0ebc3bbfce8bcca25"}
Jul 17 17:34:27.551615 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:27.551548    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l" podStartSLOduration=1.3664608870000001 podStartE2EDuration="2.551527222s" totalImagesPullingTime="1.185066335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:34:27.54939155 +0000 UTC m=+639.554747415" watchObservedRunningTime="2026-07-17 17:34:27.551527222 +0000 UTC m=+639.556883146"
Jul 17 17:34:35.698483 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:35.698430    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:35.699026 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:35.698565    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:35.711011 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:35.710980    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:34:36.585948 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:34:36.585916    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l"
Jul 17 17:35:32.490193 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.490155    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"]
Jul 17 17:35:32.492755 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.492726    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.495691 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.495666    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 17 17:35:32.503949 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.503920    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"]
Jul 17 17:35:32.578292 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.578258    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.578548 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.578301    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.578548 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.578352    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.578548 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.578389    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.578548 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.578439    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70cf4691-f39b-445d-ab1a-0e04dcce50d5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.578735 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.578645    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77vdm\" (UniqueName: \"kubernetes.io/projected/70cf4691-f39b-445d-ab1a-0e04dcce50d5-kube-api-access-77vdm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.679412 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.679359    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77vdm\" (UniqueName: \"kubernetes.io/projected/70cf4691-f39b-445d-ab1a-0e04dcce50d5-kube-api-access-77vdm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.679689 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.679475    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.679689 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.679532    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.679689 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.679571    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.679689 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.679607    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.679689 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.679630    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70cf4691-f39b-445d-ab1a-0e04dcce50d5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.680039 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.680011    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.680097 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.680020    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.680097 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.680056    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.682032 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.682006    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70cf4691-f39b-445d-ab1a-0e04dcce50d5-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.682184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.682164    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70cf4691-f39b-445d-ab1a-0e04dcce50d5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.688262 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.688224    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77vdm\" (UniqueName: \"kubernetes.io/projected/70cf4691-f39b-445d-ab1a-0e04dcce50d5-kube-api-access-77vdm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j\" (UID: \"70cf4691-f39b-445d-ab1a-0e04dcce50d5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.802937 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.802839    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:32.948351 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:32.948326    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"]
Jul 17 17:35:32.950235 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:35:32.950208    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70cf4691_f39b_445d_ab1a_0e04dcce50d5.slice/crio-ae62b5c32e47130fc2e342d2868ffac4656732c4f8324e8045f1a1c66879e0c4 WatchSource:0}: Error finding container ae62b5c32e47130fc2e342d2868ffac4656732c4f8324e8045f1a1c66879e0c4: Status 404 returned error can't find the container with id ae62b5c32e47130fc2e342d2868ffac4656732c4f8324e8045f1a1c66879e0c4
Jul 17 17:35:33.887557 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:33.887520    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j" event={"ID":"70cf4691-f39b-445d-ab1a-0e04dcce50d5","Type":"ContainerStarted","Data":"985b0dda75fbc3c1952d8331a537f175e0ea1a65381b4cb8cb5b7a12531d1217"}
Jul 17 17:35:33.887981 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:33.887566    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j" event={"ID":"70cf4691-f39b-445d-ab1a-0e04dcce50d5","Type":"ContainerStarted","Data":"ae62b5c32e47130fc2e342d2868ffac4656732c4f8324e8045f1a1c66879e0c4"}
Jul 17 17:35:33.907022 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:33.906962    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j" podStartSLOduration=1.730885027 podStartE2EDuration="1.906941503s" totalImagesPullingTime="176.056476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:35:33.90503682 +0000 UTC m=+705.910392701" watchObservedRunningTime="2026-07-17 17:35:33.906941503 +0000 UTC m=+705.912297357"
Jul 17 17:35:42.803464 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:42.803363    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:42.803464 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:42.803425    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:42.816201 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:42.816166    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:35:42.945138 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:35:42.945109    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j"
Jul 17 17:38:48.491869 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:38:48.491787    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:38:48.493211 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:38:48.493189    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:38:48.499285 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:38:48.499262    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:38:48.500027 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:38:48.500006    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:43:48.527950 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:43:48.527869    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:43:48.528539 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:43:48.528243    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:43:48.534167 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:43:48.534143    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:43:48.534434 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:43:48.534416    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:44:05.790945 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:05.790906    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"]
Jul 17 17:44:05.791547 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:05.791135    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx" podUID="346eaea9-35bf-4d22-ac45-7074a0617994" containerName="manager" containerID="cri-o://1ea01b827b7918127c6c7529a883e26bdaed87fd098657e8dd207008dacdce74" gracePeriod=10
Jul 17 17:44:06.554931 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:06.554889    2572 generic.go:356] "Generic (PLEG): container finished" podID="346eaea9-35bf-4d22-ac45-7074a0617994" containerID="1ea01b827b7918127c6c7529a883e26bdaed87fd098657e8dd207008dacdce74" exitCode=0
Jul 17 17:44:06.555110 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:06.554958    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx" event={"ID":"346eaea9-35bf-4d22-ac45-7074a0617994","Type":"ContainerDied","Data":"1ea01b827b7918127c6c7529a883e26bdaed87fd098657e8dd207008dacdce74"}
Jul 17 17:44:06.649036 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:06.649005    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:44:06.784724 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:06.784635    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84s4v\" (UniqueName: \"kubernetes.io/projected/346eaea9-35bf-4d22-ac45-7074a0617994-kube-api-access-84s4v\") pod \"346eaea9-35bf-4d22-ac45-7074a0617994\" (UID: \"346eaea9-35bf-4d22-ac45-7074a0617994\") "
Jul 17 17:44:06.784724 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:06.784713    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/346eaea9-35bf-4d22-ac45-7074a0617994-extensions-socket-volume\") pod \"346eaea9-35bf-4d22-ac45-7074a0617994\" (UID: \"346eaea9-35bf-4d22-ac45-7074a0617994\") "
Jul 17 17:44:06.785143 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:06.785114    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/346eaea9-35bf-4d22-ac45-7074a0617994-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "346eaea9-35bf-4d22-ac45-7074a0617994" (UID: "346eaea9-35bf-4d22-ac45-7074a0617994"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 17:44:06.786906 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:06.786875    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/346eaea9-35bf-4d22-ac45-7074a0617994-kube-api-access-84s4v" (OuterVolumeSpecName: "kube-api-access-84s4v") pod "346eaea9-35bf-4d22-ac45-7074a0617994" (UID: "346eaea9-35bf-4d22-ac45-7074a0617994"). InnerVolumeSpecName "kube-api-access-84s4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:44:06.885975 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:06.885932    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84s4v\" (UniqueName: \"kubernetes.io/projected/346eaea9-35bf-4d22-ac45-7074a0617994-kube-api-access-84s4v\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:44:06.885975 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:06.885966    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/346eaea9-35bf-4d22-ac45-7074a0617994-extensions-socket-volume\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:44:07.561588 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:07.561560    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"
Jul 17 17:44:07.561588 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:07.561574    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx" event={"ID":"346eaea9-35bf-4d22-ac45-7074a0617994","Type":"ContainerDied","Data":"455a6e4b84fd311182df3b16573ffde174410b981ce1867167d0b5bb14c03eb5"}
Jul 17 17:44:07.561837 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:07.561635    2572 scope.go:117] "RemoveContainer" containerID="1ea01b827b7918127c6c7529a883e26bdaed87fd098657e8dd207008dacdce74"
Jul 17 17:44:07.585681 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:07.585643    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"]
Jul 17 17:44:07.595320 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:07.591448    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vr4qx"]
Jul 17 17:44:08.515917 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:44:08.515884    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="346eaea9-35bf-4d22-ac45-7074a0617994" path="/var/lib/kubelet/pods/346eaea9-35bf-4d22-ac45-7074a0617994/volumes"
Jul 17 17:45:00.146521 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.146458    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p"]
Jul 17 17:45:00.147112 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.147022    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="346eaea9-35bf-4d22-ac45-7074a0617994" containerName="manager"
Jul 17 17:45:00.147112 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.147044    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="346eaea9-35bf-4d22-ac45-7074a0617994" containerName="manager"
Jul 17 17:45:00.147244 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.147127    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="346eaea9-35bf-4d22-ac45-7074a0617994" containerName="manager"
Jul 17 17:45:00.149295 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.149273    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p"
Jul 17 17:45:00.152002 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.151977    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:45:00.157338 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.156940    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p"]
Jul 17 17:45:00.160786 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.160757    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p"
Jul 17 17:45:00.301690 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.301665    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p"]
Jul 17 17:45:00.303756 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:45:00.303726    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9557872a_ef96_4fa6_991f_eb43450860ce.slice/crio-7db237d7b725975981972e7382dbf74da3303332ab8e2170b28a01aacf5f64aa WatchSource:0}: Error finding container 7db237d7b725975981972e7382dbf74da3303332ab8e2170b28a01aacf5f64aa: Status 404 returned error can't find the container with id 7db237d7b725975981972e7382dbf74da3303332ab8e2170b28a01aacf5f64aa
Jul 17 17:45:00.305759 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.305742    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 17:45:00.837103 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:00.837060    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p" event={"ID":"9557872a-ef96-4fa6-991f-eb43450860ce","Type":"ContainerStarted","Data":"7db237d7b725975981972e7382dbf74da3303332ab8e2170b28a01aacf5f64aa"}
Jul 17 17:45:02.850264 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:02.850215    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p" event={"ID":"9557872a-ef96-4fa6-991f-eb43450860ce","Type":"ContainerStarted","Data":"2102bcdac7517db8c9c85aed4887ceddfd5661409b9a00edf97e40e00ebebfda"}
Jul 17 17:45:02.867115 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:02.867054    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p" podStartSLOduration=0.834815998 podStartE2EDuration="2.867031066s" totalImagesPullingTime="2.032215068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:45:02.866981483 +0000 UTC m=+1274.872337337" watchObservedRunningTime="2026-07-17 17:45:02.867031066 +0000 UTC m=+1274.872386918"
Jul 17 17:45:03.856624 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:03.856590    2572 generic.go:356] "Generic (PLEG): container finished" podID="9557872a-ef96-4fa6-991f-eb43450860ce" containerID="2102bcdac7517db8c9c85aed4887ceddfd5661409b9a00edf97e40e00ebebfda" exitCode=0
Jul 17 17:45:03.857016 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:03.856681    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p" event={"ID":"9557872a-ef96-4fa6-991f-eb43450860ce","Type":"ContainerDied","Data":"2102bcdac7517db8c9c85aed4887ceddfd5661409b9a00edf97e40e00ebebfda"}
Jul 17 17:45:04.994101 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:04.994071    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p"
Jul 17 17:45:05.868143 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:05.868104    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p" event={"ID":"9557872a-ef96-4fa6-991f-eb43450860ce","Type":"ContainerDied","Data":"7db237d7b725975981972e7382dbf74da3303332ab8e2170b28a01aacf5f64aa"}
Jul 17 17:45:05.868143 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:05.868142    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738505-qqc2p"
Jul 17 17:45:05.868380 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:05.868163    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7db237d7b725975981972e7382dbf74da3303332ab8e2170b28a01aacf5f64aa"
Jul 17 17:45:06.971889 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:06.971858    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"]
Jul 17 17:45:06.972311 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:06.972190    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9557872a-ef96-4fa6-991f-eb43450860ce" containerName="cleanup"
Jul 17 17:45:06.972311 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:06.972201    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9557872a-ef96-4fa6-991f-eb43450860ce" containerName="cleanup"
Jul 17 17:45:06.972311 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:06.972254    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9557872a-ef96-4fa6-991f-eb43450860ce" containerName="cleanup"
Jul 17 17:45:06.974747 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:06.974729    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:45:06.977988 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:06.977954    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 17:45:06.978241 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:06.978223    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fkc44\""
Jul 17 17:45:06.978673 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:06.978655    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 17:45:06.996424 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:06.996390    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"]
Jul 17 17:45:07.018230 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.018179    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9rqk\" (UniqueName: \"kubernetes.io/projected/60f4ff0c-da55-44d0-9a49-b74260bb4a6c-kube-api-access-b9rqk\") pod \"kuadrant-operator-controller-manager-86878c769f-t7j95\" (UID: \"60f4ff0c-da55-44d0-9a49-b74260bb4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:45:07.018412 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.018235    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60f4ff0c-da55-44d0-9a49-b74260bb4a6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t7j95\" (UID: \"60f4ff0c-da55-44d0-9a49-b74260bb4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:45:07.118827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.118783    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9rqk\" (UniqueName: \"kubernetes.io/projected/60f4ff0c-da55-44d0-9a49-b74260bb4a6c-kube-api-access-b9rqk\") pod \"kuadrant-operator-controller-manager-86878c769f-t7j95\" (UID: \"60f4ff0c-da55-44d0-9a49-b74260bb4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:45:07.118827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.118829    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60f4ff0c-da55-44d0-9a49-b74260bb4a6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t7j95\" (UID: \"60f4ff0c-da55-44d0-9a49-b74260bb4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:45:07.119215 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.119199    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60f4ff0c-da55-44d0-9a49-b74260bb4a6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t7j95\" (UID: \"60f4ff0c-da55-44d0-9a49-b74260bb4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:45:07.129252 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.129222    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9rqk\" (UniqueName: \"kubernetes.io/projected/60f4ff0c-da55-44d0-9a49-b74260bb4a6c-kube-api-access-b9rqk\") pod \"kuadrant-operator-controller-manager-86878c769f-t7j95\" (UID: \"60f4ff0c-da55-44d0-9a49-b74260bb4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:45:07.285074 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.284983    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:45:07.427691 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.427665    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"]
Jul 17 17:45:07.429760 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:45:07.429728    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f4ff0c_da55_44d0_9a49_b74260bb4a6c.slice/crio-eaa8532cabfb12a446a24016b15b959d03ef6c1db0c3bf997279295f2d26d1a1 WatchSource:0}: Error finding container eaa8532cabfb12a446a24016b15b959d03ef6c1db0c3bf997279295f2d26d1a1: Status 404 returned error can't find the container with id eaa8532cabfb12a446a24016b15b959d03ef6c1db0c3bf997279295f2d26d1a1
Jul 17 17:45:07.879434 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.879375    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95" event={"ID":"60f4ff0c-da55-44d0-9a49-b74260bb4a6c","Type":"ContainerStarted","Data":"fbfe61f1dea8ac6aad12058d89d6a7f891d5247eb1b6c99105f9d05b8ca09dd3"}
Jul 17 17:45:07.879653 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.879457    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95" event={"ID":"60f4ff0c-da55-44d0-9a49-b74260bb4a6c","Type":"ContainerStarted","Data":"eaa8532cabfb12a446a24016b15b959d03ef6c1db0c3bf997279295f2d26d1a1"}
Jul 17 17:45:07.879653 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.879484    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:45:07.899357 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:07.899300    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95" podStartSLOduration=1.899283869 podStartE2EDuration="1.899283869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:45:07.898711342 +0000 UTC m=+1279.904067211" watchObservedRunningTime="2026-07-17 17:45:07.899283869 +0000 UTC m=+1279.904639723"
Jul 17 17:45:18.887433 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:45:18.887399    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7j95"
Jul 17 17:48:48.560373 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:48:48.560338    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:48:48.562050 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:48:48.562023    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:48:48.566513 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:48:48.566473    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:48:48.568731 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:48:48.568709    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:51:28.700699 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:28.700655    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"]
Jul 17 17:51:28.703969 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:28.703951    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"
Jul 17 17:51:28.706564 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:28.706533    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:51:28.706564 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:28.706554    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:51:28.712851 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:28.712826    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"]
Jul 17 17:51:28.825342 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:28.825303    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4696465c-36fa-499f-b668-dfd86e42b294-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-kj24p\" (UID: \"4696465c-36fa-499f-b668-dfd86e42b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"
Jul 17 17:51:28.926794 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:28.926732    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4696465c-36fa-499f-b668-dfd86e42b294-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-kj24p\" (UID: \"4696465c-36fa-499f-b668-dfd86e42b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"
Jul 17 17:51:28.927385 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:28.927362    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4696465c-36fa-499f-b668-dfd86e42b294-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-kj24p\" (UID: \"4696465c-36fa-499f-b668-dfd86e42b294\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"
Jul 17 17:51:29.015463 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:29.015361    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"
Jul 17 17:51:29.163195 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:29.163152    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"]
Jul 17 17:51:29.171567 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:51:29.171479    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4696465c_36fa_499f_b668_dfd86e42b294.slice/crio-1de7ebcfe625de1753a24b6a3865600da1ebe4eed43595e1e28e254d9f4d81f4 WatchSource:0}: Error finding container 1de7ebcfe625de1753a24b6a3865600da1ebe4eed43595e1e28e254d9f4d81f4: Status 404 returned error can't find the container with id 1de7ebcfe625de1753a24b6a3865600da1ebe4eed43595e1e28e254d9f4d81f4
Jul 17 17:51:29.836272 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:29.836221    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" event={"ID":"4696465c-36fa-499f-b668-dfd86e42b294","Type":"ContainerStarted","Data":"a7e64ad07b3e4410da9570839eb760cdcbf60ec5dada4e09ca85bcbea3cbd349"}
Jul 17 17:51:29.836719 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:29.836297    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" event={"ID":"4696465c-36fa-499f-b668-dfd86e42b294","Type":"ContainerStarted","Data":"1de7ebcfe625de1753a24b6a3865600da1ebe4eed43595e1e28e254d9f4d81f4"}
Jul 17 17:51:29.854599 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:29.854535    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" podStartSLOduration=1.8545156390000002 podStartE2EDuration="1.854515639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:51:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:51:29.852278344 +0000 UTC m=+1661.857634199" watchObservedRunningTime="2026-07-17 17:51:29.854515639 +0000 UTC m=+1661.859871490"
Jul 17 17:51:30.842302 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:30.842266    2572 generic.go:356] "Generic (PLEG): container finished" podID="4696465c-36fa-499f-b668-dfd86e42b294" containerID="a7e64ad07b3e4410da9570839eb760cdcbf60ec5dada4e09ca85bcbea3cbd349" exitCode=7
Jul 17 17:51:30.842755 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:30.842323    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" event={"ID":"4696465c-36fa-499f-b668-dfd86e42b294","Type":"ContainerDied","Data":"a7e64ad07b3e4410da9570839eb760cdcbf60ec5dada4e09ca85bcbea3cbd349"}
Jul 17 17:51:30.842755 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:30.842567    2572 scope.go:117] "RemoveContainer" containerID="a7e64ad07b3e4410da9570839eb760cdcbf60ec5dada4e09ca85bcbea3cbd349"
Jul 17 17:51:31.849480 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:31.849445    2572 generic.go:356] "Generic (PLEG): container finished" podID="4696465c-36fa-499f-b668-dfd86e42b294" containerID="247322d53580bc6d51b9344eb56ce266f804384acc89c85bdabf3fdf301f77e0" exitCode=7
Jul 17 17:51:31.849984 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:31.849524    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" event={"ID":"4696465c-36fa-499f-b668-dfd86e42b294","Type":"ContainerDied","Data":"247322d53580bc6d51b9344eb56ce266f804384acc89c85bdabf3fdf301f77e0"}
Jul 17 17:51:31.849984 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:31.849583    2572 scope.go:117] "RemoveContainer" containerID="a7e64ad07b3e4410da9570839eb760cdcbf60ec5dada4e09ca85bcbea3cbd349"
Jul 17 17:51:31.849984 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:31.849776    2572 scope.go:117] "RemoveContainer" containerID="247322d53580bc6d51b9344eb56ce266f804384acc89c85bdabf3fdf301f77e0"
Jul 17 17:51:31.850154 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:51:31.849990    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-kj24p_odh-ai-gateway-infra(4696465c-36fa-499f-b668-dfd86e42b294)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" podUID="4696465c-36fa-499f-b668-dfd86e42b294"
Jul 17 17:51:32.857125 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:32.857103    2572 scope.go:117] "RemoveContainer" containerID="247322d53580bc6d51b9344eb56ce266f804384acc89c85bdabf3fdf301f77e0"
Jul 17 17:51:32.857541 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:51:32.857294    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-kj24p_odh-ai-gateway-infra(4696465c-36fa-499f-b668-dfd86e42b294)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" podUID="4696465c-36fa-499f-b668-dfd86e42b294"
Jul 17 17:51:43.510134 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:43.510096    2572 scope.go:117] "RemoveContainer" containerID="247322d53580bc6d51b9344eb56ce266f804384acc89c85bdabf3fdf301f77e0"
Jul 17 17:51:43.914646 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:43.914611    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" event={"ID":"4696465c-36fa-499f-b668-dfd86e42b294","Type":"ContainerStarted","Data":"76a7903024afc857eeea8dbfdaeb08894b575c1166696011f818bb062931d9dd"}
Jul 17 17:51:44.921636 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:44.921603    2572 generic.go:356] "Generic (PLEG): container finished" podID="4696465c-36fa-499f-b668-dfd86e42b294" containerID="76a7903024afc857eeea8dbfdaeb08894b575c1166696011f818bb062931d9dd" exitCode=7
Jul 17 17:51:44.922148 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:44.921672    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" event={"ID":"4696465c-36fa-499f-b668-dfd86e42b294","Type":"ContainerDied","Data":"76a7903024afc857eeea8dbfdaeb08894b575c1166696011f818bb062931d9dd"}
Jul 17 17:51:44.922148 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:44.921744    2572 scope.go:117] "RemoveContainer" containerID="247322d53580bc6d51b9344eb56ce266f804384acc89c85bdabf3fdf301f77e0"
Jul 17 17:51:44.922148 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:44.922064    2572 scope.go:117] "RemoveContainer" containerID="76a7903024afc857eeea8dbfdaeb08894b575c1166696011f818bb062931d9dd"
Jul 17 17:51:44.922363 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:51:44.922341    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-kj24p_odh-ai-gateway-infra(4696465c-36fa-499f-b668-dfd86e42b294)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" podUID="4696465c-36fa-499f-b668-dfd86e42b294"
Jul 17 17:51:44.944444 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:44.944395    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"]
Jul 17 17:51:46.067974 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:46.067942    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"
Jul 17 17:51:46.181515 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:46.181396    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4696465c-36fa-499f-b668-dfd86e42b294-maas-api-service-ca\") pod \"4696465c-36fa-499f-b668-dfd86e42b294\" (UID: \"4696465c-36fa-499f-b668-dfd86e42b294\") "
Jul 17 17:51:46.181846 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:46.181818    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4696465c-36fa-499f-b668-dfd86e42b294-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4696465c-36fa-499f-b668-dfd86e42b294" (UID: "4696465c-36fa-499f-b668-dfd86e42b294"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:51:46.282336 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:46.282299    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4696465c-36fa-499f-b668-dfd86e42b294-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:51:46.934977 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:46.934942    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"
Jul 17 17:51:46.935136 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:46.934937    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p" event={"ID":"4696465c-36fa-499f-b668-dfd86e42b294","Type":"ContainerDied","Data":"1de7ebcfe625de1753a24b6a3865600da1ebe4eed43595e1e28e254d9f4d81f4"}
Jul 17 17:51:46.935136 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:46.935078    2572 scope.go:117] "RemoveContainer" containerID="76a7903024afc857eeea8dbfdaeb08894b575c1166696011f818bb062931d9dd"
Jul 17 17:51:46.953201 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:46.953168    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"]
Jul 17 17:51:46.967449 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:46.967418    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-kj24p"]
Jul 17 17:51:48.516272 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:51:48.516237    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4696465c-36fa-499f-b668-dfd86e42b294" path="/var/lib/kubelet/pods/4696465c-36fa-499f-b668-dfd86e42b294/volumes"
Jul 17 17:52:18.745422 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.745337    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"]
Jul 17 17:52:18.745899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.745690    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4696465c-36fa-499f-b668-dfd86e42b294" containerName="revoke-keys"
Jul 17 17:52:18.745899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.745702    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4696465c-36fa-499f-b668-dfd86e42b294" containerName="revoke-keys"
Jul 17 17:52:18.745899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.745724    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4696465c-36fa-499f-b668-dfd86e42b294" containerName="revoke-keys"
Jul 17 17:52:18.745899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.745729    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4696465c-36fa-499f-b668-dfd86e42b294" containerName="revoke-keys"
Jul 17 17:52:18.745899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.745784    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4696465c-36fa-499f-b668-dfd86e42b294" containerName="revoke-keys"
Jul 17 17:52:18.745899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.745793    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4696465c-36fa-499f-b668-dfd86e42b294" containerName="revoke-keys"
Jul 17 17:52:18.745899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.745800    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4696465c-36fa-499f-b668-dfd86e42b294" containerName="revoke-keys"
Jul 17 17:52:18.748865 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.748842    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"
Jul 17 17:52:18.751643 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.751613    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:52:18.751767 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.751623    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:52:18.758087 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.758025    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"]
Jul 17 17:52:18.867516 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.867452    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4f0864-4448-4082-a867-a38e432ea085-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h\" (UID: \"7a4f0864-4448-4082-a867-a38e432ea085\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"
Jul 17 17:52:18.967933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.967901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4f0864-4448-4082-a867-a38e432ea085-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h\" (UID: \"7a4f0864-4448-4082-a867-a38e432ea085\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"
Jul 17 17:52:18.968681 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:18.968657    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4f0864-4448-4082-a867-a38e432ea085-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h\" (UID: \"7a4f0864-4448-4082-a867-a38e432ea085\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"
Jul 17 17:52:19.060662 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:19.060626    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"
Jul 17 17:52:19.211176 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:19.211145    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"]
Jul 17 17:52:19.213718 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:52:19.213684    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a4f0864_4448_4082_a867_a38e432ea085.slice/crio-b01cd51c3cbecc787f0465a54a0e508a073d8e97341ce4b47d7f3de4f1c79078 WatchSource:0}: Error finding container b01cd51c3cbecc787f0465a54a0e508a073d8e97341ce4b47d7f3de4f1c79078: Status 404 returned error can't find the container with id b01cd51c3cbecc787f0465a54a0e508a073d8e97341ce4b47d7f3de4f1c79078
Jul 17 17:52:20.102147 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:20.102110    2572 generic.go:356] "Generic (PLEG): container finished" podID="7a4f0864-4448-4082-a867-a38e432ea085" containerID="57f611a70ef75441506f2deb46e975e6dae9b81189539064d219f00c9896621c" exitCode=7
Jul 17 17:52:20.102600 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:20.102203    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h" event={"ID":"7a4f0864-4448-4082-a867-a38e432ea085","Type":"ContainerDied","Data":"57f611a70ef75441506f2deb46e975e6dae9b81189539064d219f00c9896621c"}
Jul 17 17:52:20.102600 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:20.102293    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h" event={"ID":"7a4f0864-4448-4082-a867-a38e432ea085","Type":"ContainerStarted","Data":"b01cd51c3cbecc787f0465a54a0e508a073d8e97341ce4b47d7f3de4f1c79078"}
Jul 17 17:52:20.102600 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:20.102388    2572 scope.go:117] "RemoveContainer" containerID="57f611a70ef75441506f2deb46e975e6dae9b81189539064d219f00c9896621c"
Jul 17 17:52:20.845837 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:52:20.845804    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a4f0864_4448_4082_a867_a38e432ea085.slice/crio-887f98ca822ca98fb1af1ad5c59f774ef60908d4b2b1c8b1f2496d16c81360dd.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:52:21.109104 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:21.109010    2572 generic.go:356] "Generic (PLEG): container finished" podID="7a4f0864-4448-4082-a867-a38e432ea085" containerID="887f98ca822ca98fb1af1ad5c59f774ef60908d4b2b1c8b1f2496d16c81360dd" exitCode=7
Jul 17 17:52:21.109575 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:21.109102    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h" event={"ID":"7a4f0864-4448-4082-a867-a38e432ea085","Type":"ContainerDied","Data":"887f98ca822ca98fb1af1ad5c59f774ef60908d4b2b1c8b1f2496d16c81360dd"}
Jul 17 17:52:21.109575 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:21.109170    2572 scope.go:117] "RemoveContainer" containerID="57f611a70ef75441506f2deb46e975e6dae9b81189539064d219f00c9896621c"
Jul 17 17:52:21.109575 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:21.109386    2572 scope.go:117] "RemoveContainer" containerID="887f98ca822ca98fb1af1ad5c59f774ef60908d4b2b1c8b1f2496d16c81360dd"
Jul 17 17:52:21.109706 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:52:21.109646    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h_odh-ai-gateway-infra(7a4f0864-4448-4082-a867-a38e432ea085)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h" podUID="7a4f0864-4448-4082-a867-a38e432ea085"
Jul 17 17:52:22.117086 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:22.117046    2572 scope.go:117] "RemoveContainer" containerID="887f98ca822ca98fb1af1ad5c59f774ef60908d4b2b1c8b1f2496d16c81360dd"
Jul 17 17:52:22.117477 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:52:22.117239    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h_odh-ai-gateway-infra(7a4f0864-4448-4082-a867-a38e432ea085)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h" podUID="7a4f0864-4448-4082-a867-a38e432ea085"
Jul 17 17:52:35.509900 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:35.509862    2572 scope.go:117] "RemoveContainer" containerID="887f98ca822ca98fb1af1ad5c59f774ef60908d4b2b1c8b1f2496d16c81360dd"
Jul 17 17:52:36.188405 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:36.188367    2572 generic.go:356] "Generic (PLEG): container finished" podID="7a4f0864-4448-4082-a867-a38e432ea085" containerID="b8126f9006501d2af52c7a43b04a67ae0c69e83dd56c977884654f7333265dab" exitCode=7
Jul 17 17:52:36.188600 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:36.188439    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h" event={"ID":"7a4f0864-4448-4082-a867-a38e432ea085","Type":"ContainerDied","Data":"b8126f9006501d2af52c7a43b04a67ae0c69e83dd56c977884654f7333265dab"}
Jul 17 17:52:36.188600 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:36.188519    2572 scope.go:117] "RemoveContainer" containerID="887f98ca822ca98fb1af1ad5c59f774ef60908d4b2b1c8b1f2496d16c81360dd"
Jul 17 17:52:36.188832 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:36.188810    2572 scope.go:117] "RemoveContainer" containerID="b8126f9006501d2af52c7a43b04a67ae0c69e83dd56c977884654f7333265dab"
Jul 17 17:52:36.189117 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:52:36.189097    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h_odh-ai-gateway-infra(7a4f0864-4448-4082-a867-a38e432ea085)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h" podUID="7a4f0864-4448-4082-a867-a38e432ea085"
Jul 17 17:52:37.216236 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:37.216198    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"]
Jul 17 17:52:37.356614 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:37.356586    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"
Jul 17 17:52:37.535368 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:37.535270    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4f0864-4448-4082-a867-a38e432ea085-maas-api-service-ca\") pod \"7a4f0864-4448-4082-a867-a38e432ea085\" (UID: \"7a4f0864-4448-4082-a867-a38e432ea085\") "
Jul 17 17:52:37.535696 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:37.535673    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a4f0864-4448-4082-a867-a38e432ea085-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7a4f0864-4448-4082-a867-a38e432ea085" (UID: "7a4f0864-4448-4082-a867-a38e432ea085"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:52:37.636967 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:37.636926    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4f0864-4448-4082-a867-a38e432ea085-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:52:38.206758 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:38.206719    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h" event={"ID":"7a4f0864-4448-4082-a867-a38e432ea085","Type":"ContainerDied","Data":"b01cd51c3cbecc787f0465a54a0e508a073d8e97341ce4b47d7f3de4f1c79078"}
Jul 17 17:52:38.206951 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:38.206808    2572 scope.go:117] "RemoveContainer" containerID="b8126f9006501d2af52c7a43b04a67ae0c69e83dd56c977884654f7333265dab"
Jul 17 17:52:38.206951 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:38.206732    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"
Jul 17 17:52:38.232201 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:38.232168    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"]
Jul 17 17:52:38.236024 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:38.235992    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-r5j2h"]
Jul 17 17:52:38.516106 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:52:38.516020    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a4f0864-4448-4082-a867-a38e432ea085" path="/var/lib/kubelet/pods/7a4f0864-4448-4082-a867-a38e432ea085/volumes"
Jul 17 17:53:08.805541 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805499    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"]
Jul 17 17:53:08.806075 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805834    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4f0864-4448-4082-a867-a38e432ea085" containerName="revoke-keys"
Jul 17 17:53:08.806075 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805844    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4f0864-4448-4082-a867-a38e432ea085" containerName="revoke-keys"
Jul 17 17:53:08.806075 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805855    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4f0864-4448-4082-a867-a38e432ea085" containerName="revoke-keys"
Jul 17 17:53:08.806075 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805861    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4f0864-4448-4082-a867-a38e432ea085" containerName="revoke-keys"
Jul 17 17:53:08.806075 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805871    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4696465c-36fa-499f-b668-dfd86e42b294" containerName="revoke-keys"
Jul 17 17:53:08.806075 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805877    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4696465c-36fa-499f-b668-dfd86e42b294" containerName="revoke-keys"
Jul 17 17:53:08.806075 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805961    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4f0864-4448-4082-a867-a38e432ea085" containerName="revoke-keys"
Jul 17 17:53:08.806075 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805970    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4f0864-4448-4082-a867-a38e432ea085" containerName="revoke-keys"
Jul 17 17:53:08.806075 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.805976    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4f0864-4448-4082-a867-a38e432ea085" containerName="revoke-keys"
Jul 17 17:53:08.808853 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.808835    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"
Jul 17 17:53:08.811874 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.811848    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:53:08.812012 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.811849    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:53:08.819933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.819901    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"]
Jul 17 17:53:08.898137 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.898089    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d77ce602-9e5c-4b5c-85c4-55a79d988e54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-gttjg\" (UID: \"d77ce602-9e5c-4b5c-85c4-55a79d988e54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"
Jul 17 17:53:08.999204 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.999160    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d77ce602-9e5c-4b5c-85c4-55a79d988e54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-gttjg\" (UID: \"d77ce602-9e5c-4b5c-85c4-55a79d988e54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"
Jul 17 17:53:08.999855 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:08.999831    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d77ce602-9e5c-4b5c-85c4-55a79d988e54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-gttjg\" (UID: \"d77ce602-9e5c-4b5c-85c4-55a79d988e54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"
Jul 17 17:53:09.122074 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:09.122036    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"
Jul 17 17:53:09.276347 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:09.276323    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"]
Jul 17 17:53:09.278268 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:53:09.278237    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd77ce602_9e5c_4b5c_85c4_55a79d988e54.slice/crio-bc6ecafdeee63a7c9bd1405afbd4013f8e8645f02f546d26b628164a897bb61a WatchSource:0}: Error finding container bc6ecafdeee63a7c9bd1405afbd4013f8e8645f02f546d26b628164a897bb61a: Status 404 returned error can't find the container with id bc6ecafdeee63a7c9bd1405afbd4013f8e8645f02f546d26b628164a897bb61a
Jul 17 17:53:09.368004 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:09.367968    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg" event={"ID":"d77ce602-9e5c-4b5c-85c4-55a79d988e54","Type":"ContainerStarted","Data":"bc6ecafdeee63a7c9bd1405afbd4013f8e8645f02f546d26b628164a897bb61a"}
Jul 17 17:53:10.374831 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:10.374798    2572 generic.go:356] "Generic (PLEG): container finished" podID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerID="5437a2c01ed45e7b44219de10607bfb721fed74f4d3c0a07513c6d95c8010ddf" exitCode=7
Jul 17 17:53:10.375245 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:10.374860    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg" event={"ID":"d77ce602-9e5c-4b5c-85c4-55a79d988e54","Type":"ContainerDied","Data":"5437a2c01ed45e7b44219de10607bfb721fed74f4d3c0a07513c6d95c8010ddf"}
Jul 17 17:53:10.375245 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:10.375111    2572 scope.go:117] "RemoveContainer" containerID="5437a2c01ed45e7b44219de10607bfb721fed74f4d3c0a07513c6d95c8010ddf"
Jul 17 17:53:11.382848 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:11.382810    2572 generic.go:356] "Generic (PLEG): container finished" podID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerID="cbfe522133f154e9e2bee57d7c8e80596c37f7bdacdd6a6ac029a10e7738668d" exitCode=7
Jul 17 17:53:11.383357 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:11.382889    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg" event={"ID":"d77ce602-9e5c-4b5c-85c4-55a79d988e54","Type":"ContainerDied","Data":"cbfe522133f154e9e2bee57d7c8e80596c37f7bdacdd6a6ac029a10e7738668d"}
Jul 17 17:53:11.383357 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:11.382951    2572 scope.go:117] "RemoveContainer" containerID="5437a2c01ed45e7b44219de10607bfb721fed74f4d3c0a07513c6d95c8010ddf"
Jul 17 17:53:11.383357 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:11.383201    2572 scope.go:117] "RemoveContainer" containerID="cbfe522133f154e9e2bee57d7c8e80596c37f7bdacdd6a6ac029a10e7738668d"
Jul 17 17:53:11.383551 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:53:11.383457    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-gttjg_odh-ai-gateway-infra(d77ce602-9e5c-4b5c-85c4-55a79d988e54)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54"
Jul 17 17:53:12.390764 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:12.390728    2572 scope.go:117] "RemoveContainer" containerID="cbfe522133f154e9e2bee57d7c8e80596c37f7bdacdd6a6ac029a10e7738668d"
Jul 17 17:53:12.391216 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:53:12.390998    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-gttjg_odh-ai-gateway-infra(d77ce602-9e5c-4b5c-85c4-55a79d988e54)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54"
Jul 17 17:53:23.510071 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:23.510039    2572 scope.go:117] "RemoveContainer" containerID="cbfe522133f154e9e2bee57d7c8e80596c37f7bdacdd6a6ac029a10e7738668d"
Jul 17 17:53:24.456599 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:24.456510    2572 generic.go:356] "Generic (PLEG): container finished" podID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerID="0a8c8fff9ac8600b44003be89b4cf9a2a63acd758c1ec33da0c92b6f3327d173" exitCode=7
Jul 17 17:53:24.456740 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:24.456527    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg" event={"ID":"d77ce602-9e5c-4b5c-85c4-55a79d988e54","Type":"ContainerDied","Data":"0a8c8fff9ac8600b44003be89b4cf9a2a63acd758c1ec33da0c92b6f3327d173"}
Jul 17 17:53:24.456740 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:24.456658    2572 scope.go:117] "RemoveContainer" containerID="cbfe522133f154e9e2bee57d7c8e80596c37f7bdacdd6a6ac029a10e7738668d"
Jul 17 17:53:24.456970 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:24.456949    2572 scope.go:117] "RemoveContainer" containerID="0a8c8fff9ac8600b44003be89b4cf9a2a63acd758c1ec33da0c92b6f3327d173"
Jul 17 17:53:24.457208 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:53:24.457188    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-gttjg_odh-ai-gateway-infra(d77ce602-9e5c-4b5c-85c4-55a79d988e54)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54"
Jul 17 17:53:25.491003 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:25.490961    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"]
Jul 17 17:53:25.628780 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:25.628750    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"
Jul 17 17:53:25.755807 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:25.755696    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d77ce602-9e5c-4b5c-85c4-55a79d988e54-maas-api-service-ca\") pod \"d77ce602-9e5c-4b5c-85c4-55a79d988e54\" (UID: \"d77ce602-9e5c-4b5c-85c4-55a79d988e54\") "
Jul 17 17:53:25.756062 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:25.756037    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d77ce602-9e5c-4b5c-85c4-55a79d988e54-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d77ce602-9e5c-4b5c-85c4-55a79d988e54" (UID: "d77ce602-9e5c-4b5c-85c4-55a79d988e54"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:53:25.857286 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:25.857248    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d77ce602-9e5c-4b5c-85c4-55a79d988e54-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:53:26.470355 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:26.470318    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg" event={"ID":"d77ce602-9e5c-4b5c-85c4-55a79d988e54","Type":"ContainerDied","Data":"bc6ecafdeee63a7c9bd1405afbd4013f8e8645f02f546d26b628164a897bb61a"}
Jul 17 17:53:26.470355 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:26.470353    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"
Jul 17 17:53:26.470670 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:26.470386    2572 scope.go:117] "RemoveContainer" containerID="0a8c8fff9ac8600b44003be89b4cf9a2a63acd758c1ec33da0c92b6f3327d173"
Jul 17 17:53:26.495206 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:26.495170    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"]
Jul 17 17:53:26.497664 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:26.497632    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gttjg"]
Jul 17 17:53:26.516151 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:26.516121    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" path="/var/lib/kubelet/pods/d77ce602-9e5c-4b5c-85c4-55a79d988e54/volumes"
Jul 17 17:53:48.593058 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:48.593027    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:53:48.596939 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:48.596910    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:53:48.599563 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:48.599537    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:53:48.603619 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:48.603596    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:53:58.843436 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.843393    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"]
Jul 17 17:53:58.843924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.843794    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerName="revoke-keys"
Jul 17 17:53:58.843924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.843807    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerName="revoke-keys"
Jul 17 17:53:58.843924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.843821    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerName="revoke-keys"
Jul 17 17:53:58.843924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.843826    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerName="revoke-keys"
Jul 17 17:53:58.843924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.843839    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4f0864-4448-4082-a867-a38e432ea085" containerName="revoke-keys"
Jul 17 17:53:58.843924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.843845    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4f0864-4448-4082-a867-a38e432ea085" containerName="revoke-keys"
Jul 17 17:53:58.843924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.843916    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerName="revoke-keys"
Jul 17 17:53:58.843924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.843926    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerName="revoke-keys"
Jul 17 17:53:58.846805 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.846786    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"
Jul 17 17:53:58.850057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.850030    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:53:58.850199 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.850032    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:53:58.854452 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.854408    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"]
Jul 17 17:53:58.916286 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:58.916246    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g\" (UID: \"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"
Jul 17 17:53:59.017765 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:59.017721    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g\" (UID: \"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"
Jul 17 17:53:59.018452 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:59.018414    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g\" (UID: \"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"
Jul 17 17:53:59.159569 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:59.159429    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"
Jul 17 17:53:59.309677 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:59.309651    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"]
Jul 17 17:53:59.311741 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:53:59.311709    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c415f5a_583c_4d0b_a7c2_d8a4b0ec9334.slice/crio-3de6070ef9c54e8b74931127142788e882644951f315b8e100b7b09920c75800 WatchSource:0}: Error finding container 3de6070ef9c54e8b74931127142788e882644951f315b8e100b7b09920c75800: Status 404 returned error can't find the container with id 3de6070ef9c54e8b74931127142788e882644951f315b8e100b7b09920c75800
Jul 17 17:53:59.643564 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:59.643519    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" event={"ID":"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334","Type":"ContainerStarted","Data":"42590ab3a1b550223fbbc598dfe14169f55f9cd9e1ce362e8b7f223550e6e39d"}
Jul 17 17:53:59.643809 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:59.643584    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" event={"ID":"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334","Type":"ContainerStarted","Data":"3de6070ef9c54e8b74931127142788e882644951f315b8e100b7b09920c75800"}
Jul 17 17:53:59.663115 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:53:59.663054    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" podStartSLOduration=1.663037105 podStartE2EDuration="1.663037105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:53:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:53:59.6608831 +0000 UTC m=+1811.666238951" watchObservedRunningTime="2026-07-17 17:53:59.663037105 +0000 UTC m=+1811.668393038"
Jul 17 17:54:00.650381 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:00.650345    2572 generic.go:356] "Generic (PLEG): container finished" podID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerID="42590ab3a1b550223fbbc598dfe14169f55f9cd9e1ce362e8b7f223550e6e39d" exitCode=7
Jul 17 17:54:00.650832 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:00.650443    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" event={"ID":"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334","Type":"ContainerDied","Data":"42590ab3a1b550223fbbc598dfe14169f55f9cd9e1ce362e8b7f223550e6e39d"}
Jul 17 17:54:00.650832 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:00.650629    2572 scope.go:117] "RemoveContainer" containerID="42590ab3a1b550223fbbc598dfe14169f55f9cd9e1ce362e8b7f223550e6e39d"
Jul 17 17:54:01.657779 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:01.657745    2572 generic.go:356] "Generic (PLEG): container finished" podID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerID="8d1e9824fffdf1bf9ee6b4350b304b31a98563faac8aa05ff051eacef2ee53f4" exitCode=7
Jul 17 17:54:01.658227 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:01.657821    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" event={"ID":"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334","Type":"ContainerDied","Data":"8d1e9824fffdf1bf9ee6b4350b304b31a98563faac8aa05ff051eacef2ee53f4"}
Jul 17 17:54:01.658227 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:01.657886    2572 scope.go:117] "RemoveContainer" containerID="42590ab3a1b550223fbbc598dfe14169f55f9cd9e1ce362e8b7f223550e6e39d"
Jul 17 17:54:01.658227 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:01.658124    2572 scope.go:117] "RemoveContainer" containerID="8d1e9824fffdf1bf9ee6b4350b304b31a98563faac8aa05ff051eacef2ee53f4"
Jul 17 17:54:01.658358 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:54:01.658340    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g_odh-ai-gateway-infra(7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334"
Jul 17 17:54:02.665390 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:02.665361    2572 scope.go:117] "RemoveContainer" containerID="8d1e9824fffdf1bf9ee6b4350b304b31a98563faac8aa05ff051eacef2ee53f4"
Jul 17 17:54:02.665828 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:54:02.665565    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g_odh-ai-gateway-infra(7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334"
Jul 17 17:54:13.510362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:13.510326    2572 scope.go:117] "RemoveContainer" containerID="8d1e9824fffdf1bf9ee6b4350b304b31a98563faac8aa05ff051eacef2ee53f4"
Jul 17 17:54:14.731216 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:14.731174    2572 generic.go:356] "Generic (PLEG): container finished" podID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerID="a04858183e0d956533342aa5cdc887fb1afca866cd6dfe9ecab8af99d2505e78" exitCode=7
Jul 17 17:54:14.732013 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:14.731251    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" event={"ID":"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334","Type":"ContainerDied","Data":"a04858183e0d956533342aa5cdc887fb1afca866cd6dfe9ecab8af99d2505e78"}
Jul 17 17:54:14.732013 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:14.731316    2572 scope.go:117] "RemoveContainer" containerID="8d1e9824fffdf1bf9ee6b4350b304b31a98563faac8aa05ff051eacef2ee53f4"
Jul 17 17:54:14.732013 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:14.731904    2572 scope.go:117] "RemoveContainer" containerID="a04858183e0d956533342aa5cdc887fb1afca866cd6dfe9ecab8af99d2505e78"
Jul 17 17:54:14.732216 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:54:14.732193    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g_odh-ai-gateway-infra(7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334"
Jul 17 17:54:15.759732 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:15.759691    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"]
Jul 17 17:54:15.903648 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:15.903619    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"
Jul 17 17:54:15.964583 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:15.964543    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334-maas-api-service-ca\") pod \"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334\" (UID: \"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334\") "
Jul 17 17:54:15.964983 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:15.964954    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" (UID: "7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:54:16.065972 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:16.065932    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:54:16.745289 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:16.745258    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"
Jul 17 17:54:16.745481 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:16.745257    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g" event={"ID":"7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334","Type":"ContainerDied","Data":"3de6070ef9c54e8b74931127142788e882644951f315b8e100b7b09920c75800"}
Jul 17 17:54:16.745481 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:16.745395    2572 scope.go:117] "RemoveContainer" containerID="a04858183e0d956533342aa5cdc887fb1afca866cd6dfe9ecab8af99d2505e78"
Jul 17 17:54:16.763483 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:16.763449    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"]
Jul 17 17:54:16.770270 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:16.770240    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cjp9g"]
Jul 17 17:54:18.515766 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:18.515733    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" path="/var/lib/kubelet/pods/7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334/volumes"
Jul 17 17:54:47.298808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.298765    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"]
Jul 17 17:54:47.299332 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299245    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerName="revoke-keys"
Jul 17 17:54:47.299332 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299265    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerName="revoke-keys"
Jul 17 17:54:47.299332 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299285    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerName="revoke-keys"
Jul 17 17:54:47.299332 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299293    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerName="revoke-keys"
Jul 17 17:54:47.299332 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299315    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerName="revoke-keys"
Jul 17 17:54:47.299332 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299322    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerName="revoke-keys"
Jul 17 17:54:47.299332 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299331    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerName="revoke-keys"
Jul 17 17:54:47.299704 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299339    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerName="revoke-keys"
Jul 17 17:54:47.299704 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299426    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerName="revoke-keys"
Jul 17 17:54:47.299704 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299445    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d77ce602-9e5c-4b5c-85c4-55a79d988e54" containerName="revoke-keys"
Jul 17 17:54:47.299704 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.299456    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerName="revoke-keys"
Jul 17 17:54:47.302583 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.302557    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"
Jul 17 17:54:47.309738 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.309710    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:54:47.309881 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.309778    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:54:47.318611 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.318571    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"]
Jul 17 17:54:47.451992 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.451947    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff920b49-4d48-4ee8-bed4-10caf5b3f0f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2\" (UID: \"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"
Jul 17 17:54:47.552464 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.552357    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff920b49-4d48-4ee8-bed4-10caf5b3f0f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2\" (UID: \"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"
Jul 17 17:54:47.553147 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.553118    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff920b49-4d48-4ee8-bed4-10caf5b3f0f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2\" (UID: \"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"
Jul 17 17:54:47.613611 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.613566    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"
Jul 17 17:54:47.764034 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.764000    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"]
Jul 17 17:54:47.765976 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:54:47.765944    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff920b49_4d48_4ee8_bed4_10caf5b3f0f4.slice/crio-6fa6f9ddcb2c88306e1d38c673a0431707b481b95c2b9898acbf8f5e521f1345 WatchSource:0}: Error finding container 6fa6f9ddcb2c88306e1d38c673a0431707b481b95c2b9898acbf8f5e521f1345: Status 404 returned error can't find the container with id 6fa6f9ddcb2c88306e1d38c673a0431707b481b95c2b9898acbf8f5e521f1345
Jul 17 17:54:47.916954 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:47.916911    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" event={"ID":"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4","Type":"ContainerStarted","Data":"6fa6f9ddcb2c88306e1d38c673a0431707b481b95c2b9898acbf8f5e521f1345"}
Jul 17 17:54:48.883137 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:48.883104    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"]
Jul 17 17:54:48.883578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:48.883525    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c415f5a-583c-4d0b-a7c2-d8a4b0ec9334" containerName="revoke-keys"
Jul 17 17:54:48.887789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:48.887761    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"
Jul 17 17:54:48.894367 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:48.894330    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"]
Jul 17 17:54:48.924924 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:48.924823    2572 generic.go:356] "Generic (PLEG): container finished" podID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerID="9a3898a59c49d30aa814d6c5f5fa58f0245a3fafa72e76a26baca5bef175696d" exitCode=7
Jul 17 17:54:48.925083 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:48.924919    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" event={"ID":"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4","Type":"ContainerDied","Data":"9a3898a59c49d30aa814d6c5f5fa58f0245a3fafa72e76a26baca5bef175696d"}
Jul 17 17:54:48.925139 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:48.925103    2572 scope.go:117] "RemoveContainer" containerID="9a3898a59c49d30aa814d6c5f5fa58f0245a3fafa72e76a26baca5bef175696d"
Jul 17 17:54:48.964636 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:48.964593    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd082ce-0953-40f8-938b-19469caf8e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq\" (UID: \"9fd082ce-0953-40f8-938b-19469caf8e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"
Jul 17 17:54:49.066132 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.066091    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd082ce-0953-40f8-938b-19469caf8e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq\" (UID: \"9fd082ce-0953-40f8-938b-19469caf8e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"
Jul 17 17:54:49.066783 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.066763    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd082ce-0953-40f8-938b-19469caf8e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq\" (UID: \"9fd082ce-0953-40f8-938b-19469caf8e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"
Jul 17 17:54:49.202344 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.202247    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"
Jul 17 17:54:49.555762 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.555732    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"]
Jul 17 17:54:49.557195 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:54:49.557168    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd082ce_0953_40f8_938b_19469caf8e5a.slice/crio-6a82a1a2f0eab51720d37717ca465008140a77dc50d253b1afb1bd7f1a469b07 WatchSource:0}: Error finding container 6a82a1a2f0eab51720d37717ca465008140a77dc50d253b1afb1bd7f1a469b07: Status 404 returned error can't find the container with id 6a82a1a2f0eab51720d37717ca465008140a77dc50d253b1afb1bd7f1a469b07
Jul 17 17:54:49.932088 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.932039    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" event={"ID":"9fd082ce-0953-40f8-938b-19469caf8e5a","Type":"ContainerStarted","Data":"c8fc01f0ff5ba39122d5b69ec17a227e0cbf2a07c2ba7a59eb67e91de3dc1221"}
Jul 17 17:54:49.932637 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.932113    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" event={"ID":"9fd082ce-0953-40f8-938b-19469caf8e5a","Type":"ContainerStarted","Data":"6a82a1a2f0eab51720d37717ca465008140a77dc50d253b1afb1bd7f1a469b07"}
Jul 17 17:54:49.934166 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.934133    2572 generic.go:356] "Generic (PLEG): container finished" podID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerID="a06c4d031691fafc1673d5e00d54e781fdda913af6bcd241789ea310b048b433" exitCode=7
Jul 17 17:54:49.934320 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.934221    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" event={"ID":"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4","Type":"ContainerDied","Data":"a06c4d031691fafc1673d5e00d54e781fdda913af6bcd241789ea310b048b433"}
Jul 17 17:54:49.934320 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.934281    2572 scope.go:117] "RemoveContainer" containerID="9a3898a59c49d30aa814d6c5f5fa58f0245a3fafa72e76a26baca5bef175696d"
Jul 17 17:54:49.934510 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.934474    2572 scope.go:117] "RemoveContainer" containerID="a06c4d031691fafc1673d5e00d54e781fdda913af6bcd241789ea310b048b433"
Jul 17 17:54:49.934732 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:54:49.934708    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2_odh-ai-gateway-infra(ff920b49-4d48-4ee8-bed4-10caf5b3f0f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4"
Jul 17 17:54:49.951617 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:49.951562    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" podStartSLOduration=1.951542891 podStartE2EDuration="1.951542891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:54:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:54:49.949413062 +0000 UTC m=+1861.954768919" watchObservedRunningTime="2026-07-17 17:54:49.951542891 +0000 UTC m=+1861.956898806"
Jul 17 17:54:50.947961 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:50.947929    2572 scope.go:117] "RemoveContainer" containerID="a06c4d031691fafc1673d5e00d54e781fdda913af6bcd241789ea310b048b433"
Jul 17 17:54:50.948414 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:54:50.948149    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2_odh-ai-gateway-infra(ff920b49-4d48-4ee8-bed4-10caf5b3f0f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4"
Jul 17 17:54:50.949319 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:50.949296    2572 generic.go:356] "Generic (PLEG): container finished" podID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerID="c8fc01f0ff5ba39122d5b69ec17a227e0cbf2a07c2ba7a59eb67e91de3dc1221" exitCode=6
Jul 17 17:54:50.949389 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:50.949350    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" event={"ID":"9fd082ce-0953-40f8-938b-19469caf8e5a","Type":"ContainerDied","Data":"c8fc01f0ff5ba39122d5b69ec17a227e0cbf2a07c2ba7a59eb67e91de3dc1221"}
Jul 17 17:54:50.949573 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:50.949561    2572 scope.go:117] "RemoveContainer" containerID="c8fc01f0ff5ba39122d5b69ec17a227e0cbf2a07c2ba7a59eb67e91de3dc1221"
Jul 17 17:54:51.956974 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:51.956935    2572 generic.go:356] "Generic (PLEG): container finished" podID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerID="6d06032a015d27d61de27d70966b800c3adca47530173850670ea9dd63f4b742" exitCode=6
Jul 17 17:54:51.957427 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:51.957022    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" event={"ID":"9fd082ce-0953-40f8-938b-19469caf8e5a","Type":"ContainerDied","Data":"6d06032a015d27d61de27d70966b800c3adca47530173850670ea9dd63f4b742"}
Jul 17 17:54:51.957427 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:51.957078    2572 scope.go:117] "RemoveContainer" containerID="c8fc01f0ff5ba39122d5b69ec17a227e0cbf2a07c2ba7a59eb67e91de3dc1221"
Jul 17 17:54:51.957427 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:51.957271    2572 scope.go:117] "RemoveContainer" containerID="6d06032a015d27d61de27d70966b800c3adca47530173850670ea9dd63f4b742"
Jul 17 17:54:51.957586 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:54:51.957554    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq_odh-ai-gateway-infra(9fd082ce-0953-40f8-938b-19469caf8e5a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a"
Jul 17 17:54:52.964538 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:54:52.964484    2572 scope.go:117] "RemoveContainer" containerID="6d06032a015d27d61de27d70966b800c3adca47530173850670ea9dd63f4b742"
Jul 17 17:54:52.964957 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:54:52.964757    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq_odh-ai-gateway-infra(9fd082ce-0953-40f8-938b-19469caf8e5a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a"
Jul 17 17:55:04.510487 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:04.510447    2572 scope.go:117] "RemoveContainer" containerID="a06c4d031691fafc1673d5e00d54e781fdda913af6bcd241789ea310b048b433"
Jul 17 17:55:05.035301 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:05.035261    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" event={"ID":"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4","Type":"ContainerStarted","Data":"6b925da1caac2b12bb5974d1078259af9a0f1d89296306d3d22969d02b26fd27"}
Jul 17 17:55:05.052198 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:05.052132    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" podStartSLOduration=18.052113197 podStartE2EDuration="18.052113197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:55:05.051194002 +0000 UTC m=+1877.056549858" watchObservedRunningTime="2026-07-17 17:55:05.052113197 +0000 UTC m=+1877.057469172"
Jul 17 17:55:06.042284 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:06.042248    2572 generic.go:356] "Generic (PLEG): container finished" podID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerID="6b925da1caac2b12bb5974d1078259af9a0f1d89296306d3d22969d02b26fd27" exitCode=7
Jul 17 17:55:06.042752 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:06.042338    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" event={"ID":"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4","Type":"ContainerDied","Data":"6b925da1caac2b12bb5974d1078259af9a0f1d89296306d3d22969d02b26fd27"}
Jul 17 17:55:06.042752 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:06.042453    2572 scope.go:117] "RemoveContainer" containerID="a06c4d031691fafc1673d5e00d54e781fdda913af6bcd241789ea310b048b433"
Jul 17 17:55:06.042877 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:06.042763    2572 scope.go:117] "RemoveContainer" containerID="6b925da1caac2b12bb5974d1078259af9a0f1d89296306d3d22969d02b26fd27"
Jul 17 17:55:06.043008 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:55:06.042982    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2_odh-ai-gateway-infra(ff920b49-4d48-4ee8-bed4-10caf5b3f0f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4"
Jul 17 17:55:06.068869 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:06.068838    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"]
Jul 17 17:55:06.517692 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:06.517653    2572 scope.go:117] "RemoveContainer" containerID="6d06032a015d27d61de27d70966b800c3adca47530173850670ea9dd63f4b742"
Jul 17 17:55:07.051411 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:07.051378    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" event={"ID":"9fd082ce-0953-40f8-938b-19469caf8e5a","Type":"ContainerStarted","Data":"bb21997cac41d3f59498501680f20c6cba1868c9ec6753c330f378487e082204"}
Jul 17 17:55:07.216250 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:07.216224    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"
Jul 17 17:55:07.329899 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:07.329850    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff920b49-4d48-4ee8-bed4-10caf5b3f0f4-maas-api-service-ca\") pod \"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4\" (UID: \"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4\") "
Jul 17 17:55:07.330278 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:07.330257    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff920b49-4d48-4ee8-bed4-10caf5b3f0f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" (UID: "ff920b49-4d48-4ee8-bed4-10caf5b3f0f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:55:07.431156 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:07.431054    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff920b49-4d48-4ee8-bed4-10caf5b3f0f4-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:55:08.058389 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.058351    2572 generic.go:356] "Generic (PLEG): container finished" podID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerID="bb21997cac41d3f59498501680f20c6cba1868c9ec6753c330f378487e082204" exitCode=6
Jul 17 17:55:08.058934 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.058420    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" event={"ID":"9fd082ce-0953-40f8-938b-19469caf8e5a","Type":"ContainerDied","Data":"bb21997cac41d3f59498501680f20c6cba1868c9ec6753c330f378487e082204"}
Jul 17 17:55:08.058934 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.058485    2572 scope.go:117] "RemoveContainer" containerID="6d06032a015d27d61de27d70966b800c3adca47530173850670ea9dd63f4b742"
Jul 17 17:55:08.058934 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.058857    2572 scope.go:117] "RemoveContainer" containerID="bb21997cac41d3f59498501680f20c6cba1868c9ec6753c330f378487e082204"
Jul 17 17:55:08.059129 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:55:08.059094    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq_odh-ai-gateway-infra(9fd082ce-0953-40f8-938b-19469caf8e5a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a"
Jul 17 17:55:08.059789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.059714    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2" event={"ID":"ff920b49-4d48-4ee8-bed4-10caf5b3f0f4","Type":"ContainerDied","Data":"6fa6f9ddcb2c88306e1d38c673a0431707b481b95c2b9898acbf8f5e521f1345"}
Jul 17 17:55:08.059789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.059740    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"
Jul 17 17:55:08.070573 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.070551    2572 scope.go:117] "RemoveContainer" containerID="6b925da1caac2b12bb5974d1078259af9a0f1d89296306d3d22969d02b26fd27"
Jul 17 17:55:08.089265 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.089228    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"]
Jul 17 17:55:08.093858 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.093767    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"]
Jul 17 17:55:08.096063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.096032    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-vzfs2"]
Jul 17 17:55:08.516380 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:08.516291    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" path="/var/lib/kubelet/pods/ff920b49-4d48-4ee8-bed4-10caf5b3f0f4/volumes"
Jul 17 17:55:09.206838 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:09.206807    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"
Jul 17 17:55:09.347514 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:09.347387    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd082ce-0953-40f8-938b-19469caf8e5a-maas-api-service-ca\") pod \"9fd082ce-0953-40f8-938b-19469caf8e5a\" (UID: \"9fd082ce-0953-40f8-938b-19469caf8e5a\") "
Jul 17 17:55:09.347799 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:09.347775    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fd082ce-0953-40f8-938b-19469caf8e5a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9fd082ce-0953-40f8-938b-19469caf8e5a" (UID: "9fd082ce-0953-40f8-938b-19469caf8e5a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:55:09.448811 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:09.448772    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd082ce-0953-40f8-938b-19469caf8e5a-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:55:10.073933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:10.073899    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"
Jul 17 17:55:10.074150 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:10.073927    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq" event={"ID":"9fd082ce-0953-40f8-938b-19469caf8e5a","Type":"ContainerDied","Data":"6a82a1a2f0eab51720d37717ca465008140a77dc50d253b1afb1bd7f1a469b07"}
Jul 17 17:55:10.074150 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:10.073989    2572 scope.go:117] "RemoveContainer" containerID="bb21997cac41d3f59498501680f20c6cba1868c9ec6753c330f378487e082204"
Jul 17 17:55:10.095996 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:10.095958    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"]
Jul 17 17:55:10.099399 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:10.099368    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-tbwrq"]
Jul 17 17:55:10.515447 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:10.515337    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" path="/var/lib/kubelet/pods/9fd082ce-0953-40f8-938b-19469caf8e5a/volumes"
Jul 17 17:55:47.415258 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415213    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"]
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415576    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415588    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415609    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415615    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415633    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415639    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415646    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415652    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415659    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415664    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415721    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415727    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415735    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415741    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerName="revoke-keys"
Jul 17 17:55:47.415789 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.415747    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerName="revoke-keys"
Jul 17 17:55:47.418772 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.418740    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"
Jul 17 17:55:47.423243 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.423218    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:55:47.423589 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.423571    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:55:47.443161 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.443123    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"]
Jul 17 17:55:47.482043 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.482005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fff5c4-f544-4e45-8a73-f852ca5d55e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg\" (UID: \"93fff5c4-f544-4e45-8a73-f852ca5d55e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"
Jul 17 17:55:47.583200 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.583145    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fff5c4-f544-4e45-8a73-f852ca5d55e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg\" (UID: \"93fff5c4-f544-4e45-8a73-f852ca5d55e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"
Jul 17 17:55:47.584477 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.584450    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fff5c4-f544-4e45-8a73-f852ca5d55e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg\" (UID: \"93fff5c4-f544-4e45-8a73-f852ca5d55e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"
Jul 17 17:55:47.730269 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.730173    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"
Jul 17 17:55:47.881151 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:47.881125    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"]
Jul 17 17:55:47.883063 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:55:47.883034    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-deb615fccd9f3969c3430636d64eb8646e8d9d075a7e0ad142da3b4dcd47d637 WatchSource:0}: Error finding container deb615fccd9f3969c3430636d64eb8646e8d9d075a7e0ad142da3b4dcd47d637: Status 404 returned error can't find the container with id deb615fccd9f3969c3430636d64eb8646e8d9d075a7e0ad142da3b4dcd47d637
Jul 17 17:55:48.282264 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.282164    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" event={"ID":"93fff5c4-f544-4e45-8a73-f852ca5d55e8","Type":"ContainerStarted","Data":"bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603"}
Jul 17 17:55:48.282264 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.282225    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" event={"ID":"93fff5c4-f544-4e45-8a73-f852ca5d55e8","Type":"ContainerStarted","Data":"deb615fccd9f3969c3430636d64eb8646e8d9d075a7e0ad142da3b4dcd47d637"}
Jul 17 17:55:48.301182 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.301117    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" podStartSLOduration=1.301095852 podStartE2EDuration="1.301095852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:55:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:55:48.29859802 +0000 UTC m=+1920.303953895" watchObservedRunningTime="2026-07-17 17:55:48.301095852 +0000 UTC m=+1920.306451706"
Jul 17 17:55:48.928783 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.928748    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"]
Jul 17 17:55:48.929184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.929116    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerName="revoke-keys"
Jul 17 17:55:48.929184 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.929129    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff920b49-4d48-4ee8-bed4-10caf5b3f0f4" containerName="revoke-keys"
Jul 17 17:55:48.929260 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.929191    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fd082ce-0953-40f8-938b-19469caf8e5a" containerName="revoke-keys"
Jul 17 17:55:48.933529 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.933482    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"
Jul 17 17:55:48.940225 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.940115    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"]
Jul 17 17:55:48.996261 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:48.996217    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a86243d-a540-4ba9-b22d-ea38ff8a3fcd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-sg258\" (UID: \"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"
Jul 17 17:55:49.096839 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:49.096796    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a86243d-a540-4ba9-b22d-ea38ff8a3fcd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-sg258\" (UID: \"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"
Jul 17 17:55:49.097452 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:49.097429    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a86243d-a540-4ba9-b22d-ea38ff8a3fcd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-sg258\" (UID: \"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"
Jul 17 17:55:49.246276 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:49.246183    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"
Jul 17 17:55:49.289140 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:49.289100    2572 generic.go:356] "Generic (PLEG): container finished" podID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerID="bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603" exitCode=7
Jul 17 17:55:49.289442 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:49.289157    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" event={"ID":"93fff5c4-f544-4e45-8a73-f852ca5d55e8","Type":"ContainerDied","Data":"bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603"}
Jul 17 17:55:49.289895 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:49.289875    2572 scope.go:117] "RemoveContainer" containerID="bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603"
Jul 17 17:55:49.397842 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:49.397806    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"]
Jul 17 17:55:49.399771 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:55:49.399740    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a86243d_a540_4ba9_b22d_ea38ff8a3fcd.slice/crio-5729b5e76ef323971dd4692c3d292602ea03075498cc2660848c2c28b25e819c WatchSource:0}: Error finding container 5729b5e76ef323971dd4692c3d292602ea03075498cc2660848c2c28b25e819c: Status 404 returned error can't find the container with id 5729b5e76ef323971dd4692c3d292602ea03075498cc2660848c2c28b25e819c
Jul 17 17:55:50.296624 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:50.296589    2572 generic.go:356] "Generic (PLEG): container finished" podID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerID="3474f3db6b0da19f1803a207c8f71d66dee2eb22b72888085e550a99aaf78c8f" exitCode=7
Jul 17 17:55:50.297096 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:50.296681    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" event={"ID":"93fff5c4-f544-4e45-8a73-f852ca5d55e8","Type":"ContainerDied","Data":"3474f3db6b0da19f1803a207c8f71d66dee2eb22b72888085e550a99aaf78c8f"}
Jul 17 17:55:50.297096 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:50.296744    2572 scope.go:117] "RemoveContainer" containerID="bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603"
Jul 17 17:55:50.297096 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:50.296942    2572 scope.go:117] "RemoveContainer" containerID="3474f3db6b0da19f1803a207c8f71d66dee2eb22b72888085e550a99aaf78c8f"
Jul 17 17:55:50.297275 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:55:50.297167    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg_odh-ai-gateway-infra(93fff5c4-f544-4e45-8a73-f852ca5d55e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8"
Jul 17 17:55:50.298378 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:50.298355    2572 generic.go:356] "Generic (PLEG): container finished" podID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerID="0941f6357b21b335f953b588b539947bd11424fe64688b5e0f83e17adeaa8362" exitCode=6
Jul 17 17:55:50.298525 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:50.298445    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258" event={"ID":"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd","Type":"ContainerDied","Data":"0941f6357b21b335f953b588b539947bd11424fe64688b5e0f83e17adeaa8362"}
Jul 17 17:55:50.298593 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:50.298529    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258" event={"ID":"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd","Type":"ContainerStarted","Data":"5729b5e76ef323971dd4692c3d292602ea03075498cc2660848c2c28b25e819c"}
Jul 17 17:55:50.298685 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:50.298667    2572 scope.go:117] "RemoveContainer" containerID="0941f6357b21b335f953b588b539947bd11424fe64688b5e0f83e17adeaa8362"
Jul 17 17:55:50.846946 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:55:50.846899    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:55:51.305635 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:51.305616    2572 scope.go:117] "RemoveContainer" containerID="3474f3db6b0da19f1803a207c8f71d66dee2eb22b72888085e550a99aaf78c8f"
Jul 17 17:55:51.306045 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:55:51.305841    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg_odh-ai-gateway-infra(93fff5c4-f544-4e45-8a73-f852ca5d55e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8"
Jul 17 17:55:51.307232 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:51.307201    2572 generic.go:356] "Generic (PLEG): container finished" podID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerID="007ca5fa944daf1ea1d9a6a82e80dcf2c2d1c32dfd328e3333c4b7d4708215a8" exitCode=6
Jul 17 17:55:51.307352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:51.307264    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258" event={"ID":"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd","Type":"ContainerDied","Data":"007ca5fa944daf1ea1d9a6a82e80dcf2c2d1c32dfd328e3333c4b7d4708215a8"}
Jul 17 17:55:51.307352 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:51.307311    2572 scope.go:117] "RemoveContainer" containerID="0941f6357b21b335f953b588b539947bd11424fe64688b5e0f83e17adeaa8362"
Jul 17 17:55:51.307546 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:51.307529    2572 scope.go:117] "RemoveContainer" containerID="007ca5fa944daf1ea1d9a6a82e80dcf2c2d1c32dfd328e3333c4b7d4708215a8"
Jul 17 17:55:51.307760 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:55:51.307737    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-sg258_odh-ai-gateway-infra(6a86243d-a540-4ba9-b22d-ea38ff8a3fcd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd"
Jul 17 17:55:52.315456 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:55:52.315420    2572 scope.go:117] "RemoveContainer" containerID="007ca5fa944daf1ea1d9a6a82e80dcf2c2d1c32dfd328e3333c4b7d4708215a8"
Jul 17 17:55:52.315878 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:55:52.315715    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-sg258_odh-ai-gateway-infra(6a86243d-a540-4ba9-b22d-ea38ff8a3fcd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd"
Jul 17 17:55:54.009546 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:55:54.006343    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:04.052958 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:04.052921    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:04.510334 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:04.510297    2572 scope.go:117] "RemoveContainer" containerID="3474f3db6b0da19f1803a207c8f71d66dee2eb22b72888085e550a99aaf78c8f"
Jul 17 17:56:05.385423 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:05.385388    2572 generic.go:356] "Generic (PLEG): container finished" podID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerID="16cd55b9874a95a7883295ffe6994ed81c75a28d9f5b35c0a1e4851983e14e37" exitCode=6
Jul 17 17:56:05.385953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:05.385467    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" event={"ID":"93fff5c4-f544-4e45-8a73-f852ca5d55e8","Type":"ContainerDied","Data":"16cd55b9874a95a7883295ffe6994ed81c75a28d9f5b35c0a1e4851983e14e37"}
Jul 17 17:56:05.385953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:05.385563    2572 scope.go:117] "RemoveContainer" containerID="3474f3db6b0da19f1803a207c8f71d66dee2eb22b72888085e550a99aaf78c8f"
Jul 17 17:56:05.385953 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:05.385895    2572 scope.go:117] "RemoveContainer" containerID="16cd55b9874a95a7883295ffe6994ed81c75a28d9f5b35c0a1e4851983e14e37"
Jul 17 17:56:05.386173 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:05.386151    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg_odh-ai-gateway-infra(93fff5c4-f544-4e45-8a73-f852ca5d55e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8"
Jul 17 17:56:05.810967 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:05.810925    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:05.869312 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:05.869276    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"]
Jul 17 17:56:05.873844 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:05.873821    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"
Jul 17 17:56:05.882065 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:05.881380    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"]
Jul 17 17:56:05.953449 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:05.953410    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c75cd2-db42-43fa-8791-5ffafde80558-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-flkk4\" (UID: \"98c75cd2-db42-43fa-8791-5ffafde80558\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"
Jul 17 17:56:06.054169 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.054123    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c75cd2-db42-43fa-8791-5ffafde80558-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-flkk4\" (UID: \"98c75cd2-db42-43fa-8791-5ffafde80558\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"
Jul 17 17:56:06.054844 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.054821    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c75cd2-db42-43fa-8791-5ffafde80558-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-flkk4\" (UID: \"98c75cd2-db42-43fa-8791-5ffafde80558\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"
Jul 17 17:56:06.186189 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.186094    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"
Jul 17 17:56:06.329930 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.329895    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"]
Jul 17 17:56:06.331639 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:56:06.331609    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c75cd2_db42_43fa_8791_5ffafde80558.slice/crio-194c946cdd8deaea9b67e769701358db44dce7537d115ea5f001addf3821f285 WatchSource:0}: Error finding container 194c946cdd8deaea9b67e769701358db44dce7537d115ea5f001addf3821f285: Status 404 returned error can't find the container with id 194c946cdd8deaea9b67e769701358db44dce7537d115ea5f001addf3821f285
Jul 17 17:56:06.392200 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.392162    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4" event={"ID":"98c75cd2-db42-43fa-8791-5ffafde80558","Type":"ContainerStarted","Data":"194c946cdd8deaea9b67e769701358db44dce7537d115ea5f001addf3821f285"}
Jul 17 17:56:06.416618 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.416571    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"]
Jul 17 17:56:06.510109 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.510075    2572 scope.go:117] "RemoveContainer" containerID="007ca5fa944daf1ea1d9a6a82e80dcf2c2d1c32dfd328e3333c4b7d4708215a8"
Jul 17 17:56:06.556610 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.556574    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"
Jul 17 17:56:06.657455 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.657417    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fff5c4-f544-4e45-8a73-f852ca5d55e8-maas-api-service-ca\") pod \"93fff5c4-f544-4e45-8a73-f852ca5d55e8\" (UID: \"93fff5c4-f544-4e45-8a73-f852ca5d55e8\") "
Jul 17 17:56:06.657894 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.657868    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93fff5c4-f544-4e45-8a73-f852ca5d55e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "93fff5c4-f544-4e45-8a73-f852ca5d55e8" (UID: "93fff5c4-f544-4e45-8a73-f852ca5d55e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:56:06.758759 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:06.758716    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fff5c4-f544-4e45-8a73-f852ca5d55e8-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:56:07.400824 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.400739    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"
Jul 17 17:56:07.400824 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.400782    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg" event={"ID":"93fff5c4-f544-4e45-8a73-f852ca5d55e8","Type":"ContainerDied","Data":"deb615fccd9f3969c3430636d64eb8646e8d9d075a7e0ad142da3b4dcd47d637"}
Jul 17 17:56:07.401284 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.400843    2572 scope.go:117] "RemoveContainer" containerID="16cd55b9874a95a7883295ffe6994ed81c75a28d9f5b35c0a1e4851983e14e37"
Jul 17 17:56:07.402906 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.402876    2572 generic.go:356] "Generic (PLEG): container finished" podID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerID="c76d6f904cd8b5144d2affd369c0fd324b5492d2b4c2e5e1c5e401d79e732e99" exitCode=6
Jul 17 17:56:07.403019 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.402953    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258" event={"ID":"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd","Type":"ContainerDied","Data":"c76d6f904cd8b5144d2affd369c0fd324b5492d2b4c2e5e1c5e401d79e732e99"}
Jul 17 17:56:07.403342 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.403323    2572 scope.go:117] "RemoveContainer" containerID="c76d6f904cd8b5144d2affd369c0fd324b5492d2b4c2e5e1c5e401d79e732e99"
Jul 17 17:56:07.403590 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:07.403565    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-sg258_odh-ai-gateway-infra(6a86243d-a540-4ba9-b22d-ea38ff8a3fcd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd"
Jul 17 17:56:07.404798 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.404758    2572 generic.go:356] "Generic (PLEG): container finished" podID="98c75cd2-db42-43fa-8791-5ffafde80558" containerID="5e6f3655e36f03412f0a242895f1773a6cac323bf1eb63da014607abc879d0c1" exitCode=7
Jul 17 17:56:07.404798 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.404785    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4" event={"ID":"98c75cd2-db42-43fa-8791-5ffafde80558","Type":"ContainerDied","Data":"5e6f3655e36f03412f0a242895f1773a6cac323bf1eb63da014607abc879d0c1"}
Jul 17 17:56:07.405034 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.405018    2572 scope.go:117] "RemoveContainer" containerID="5e6f3655e36f03412f0a242895f1773a6cac323bf1eb63da014607abc879d0c1"
Jul 17 17:56:07.411072 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.411055    2572 scope.go:117] "RemoveContainer" containerID="007ca5fa944daf1ea1d9a6a82e80dcf2c2d1c32dfd328e3333c4b7d4708215a8"
Jul 17 17:56:07.452428 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.452396    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"]
Jul 17 17:56:07.456144 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:07.456112    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6qspg"]
Jul 17 17:56:08.413011 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.412976    2572 generic.go:356] "Generic (PLEG): container finished" podID="98c75cd2-db42-43fa-8791-5ffafde80558" containerID="874cfd167e5affca2fbbecd6d7d32862ccca785dfbb4014aa207568ee0e903bb" exitCode=7
Jul 17 17:56:08.413551 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.413051    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4" event={"ID":"98c75cd2-db42-43fa-8791-5ffafde80558","Type":"ContainerDied","Data":"874cfd167e5affca2fbbecd6d7d32862ccca785dfbb4014aa207568ee0e903bb"}
Jul 17 17:56:08.413551 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.413103    2572 scope.go:117] "RemoveContainer" containerID="5e6f3655e36f03412f0a242895f1773a6cac323bf1eb63da014607abc879d0c1"
Jul 17 17:56:08.413551 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.413298    2572 scope.go:117] "RemoveContainer" containerID="874cfd167e5affca2fbbecd6d7d32862ccca785dfbb4014aa207568ee0e903bb"
Jul 17 17:56:08.413732 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:08.413606    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-flkk4_odh-ai-gateway-infra(98c75cd2-db42-43fa-8791-5ffafde80558)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4" podUID="98c75cd2-db42-43fa-8791-5ffafde80558"
Jul 17 17:56:08.432306 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.432272    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"]
Jul 17 17:56:08.516342 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.516299    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" path="/var/lib/kubelet/pods/93fff5c4-f544-4e45-8a73-f852ca5d55e8/volumes"
Jul 17 17:56:08.576507 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.576468    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"
Jul 17 17:56:08.674683 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.674642    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a86243d-a540-4ba9-b22d-ea38ff8a3fcd-maas-api-service-ca\") pod \"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd\" (UID: \"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd\") "
Jul 17 17:56:08.675057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.675034    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a86243d-a540-4ba9-b22d-ea38ff8a3fcd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" (UID: "6a86243d-a540-4ba9-b22d-ea38ff8a3fcd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:56:08.775808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:08.775713    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a86243d-a540-4ba9-b22d-ea38ff8a3fcd-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:56:09.420958 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:09.420923    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"
Jul 17 17:56:09.421479 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:09.420958    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258" event={"ID":"6a86243d-a540-4ba9-b22d-ea38ff8a3fcd","Type":"ContainerDied","Data":"5729b5e76ef323971dd4692c3d292602ea03075498cc2660848c2c28b25e819c"}
Jul 17 17:56:09.421479 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:09.421034    2572 scope.go:117] "RemoveContainer" containerID="c76d6f904cd8b5144d2affd369c0fd324b5492d2b4c2e5e1c5e401d79e732e99"
Jul 17 17:56:09.423190 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:09.423168    2572 scope.go:117] "RemoveContainer" containerID="874cfd167e5affca2fbbecd6d7d32862ccca785dfbb4014aa207568ee0e903bb"
Jul 17 17:56:09.423446 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:09.423423    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-flkk4_odh-ai-gateway-infra(98c75cd2-db42-43fa-8791-5ffafde80558)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4" podUID="98c75cd2-db42-43fa-8791-5ffafde80558"
Jul 17 17:56:09.458458 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:09.458414    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"]
Jul 17 17:56:09.463165 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:09.463126    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sg258"]
Jul 17 17:56:10.515944 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:10.515907    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" path="/var/lib/kubelet/pods/6a86243d-a540-4ba9-b22d-ea38ff8a3fcd/volumes"
Jul 17 17:56:14.096361 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:14.096318    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:20.843812 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:20.843766    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:23.510658 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:23.510628    2572 scope.go:117] "RemoveContainer" containerID="874cfd167e5affca2fbbecd6d7d32862ccca785dfbb4014aa207568ee0e903bb"
Jul 17 17:56:24.106459 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:24.106411    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:24.504128 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:24.504037    2572 generic.go:356] "Generic (PLEG): container finished" podID="98c75cd2-db42-43fa-8791-5ffafde80558" containerID="443e188ad5d9dae13e8b2f99fe26e76a5824d58852b3fedde84df9f9bd9baf6f" exitCode=7
Jul 17 17:56:24.504128 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:24.504114    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4" event={"ID":"98c75cd2-db42-43fa-8791-5ffafde80558","Type":"ContainerDied","Data":"443e188ad5d9dae13e8b2f99fe26e76a5824d58852b3fedde84df9f9bd9baf6f"}
Jul 17 17:56:24.504332 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:24.504205    2572 scope.go:117] "RemoveContainer" containerID="874cfd167e5affca2fbbecd6d7d32862ccca785dfbb4014aa207568ee0e903bb"
Jul 17 17:56:24.504592 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:24.504576    2572 scope.go:117] "RemoveContainer" containerID="443e188ad5d9dae13e8b2f99fe26e76a5824d58852b3fedde84df9f9bd9baf6f"
Jul 17 17:56:24.504842 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:24.504823    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-flkk4_odh-ai-gateway-infra(98c75cd2-db42-43fa-8791-5ffafde80558)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4" podUID="98c75cd2-db42-43fa-8791-5ffafde80558"
Jul 17 17:56:25.549811 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:25.549774    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"]
Jul 17 17:56:25.691659 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:25.691626    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"
Jul 17 17:56:25.734744 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:25.734704    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c75cd2-db42-43fa-8791-5ffafde80558-maas-api-service-ca\") pod \"98c75cd2-db42-43fa-8791-5ffafde80558\" (UID: \"98c75cd2-db42-43fa-8791-5ffafde80558\") "
Jul 17 17:56:25.735193 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:25.735161    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98c75cd2-db42-43fa-8791-5ffafde80558-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "98c75cd2-db42-43fa-8791-5ffafde80558" (UID: "98c75cd2-db42-43fa-8791-5ffafde80558"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:56:25.836109 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:25.836065    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98c75cd2-db42-43fa-8791-5ffafde80558-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:56:26.518670 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:26.518632    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4" event={"ID":"98c75cd2-db42-43fa-8791-5ffafde80558","Type":"ContainerDied","Data":"194c946cdd8deaea9b67e769701358db44dce7537d115ea5f001addf3821f285"}
Jul 17 17:56:26.518670 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:26.518663    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"
Jul 17 17:56:26.518941 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:26.518710    2572 scope.go:117] "RemoveContainer" containerID="443e188ad5d9dae13e8b2f99fe26e76a5824d58852b3fedde84df9f9bd9baf6f"
Jul 17 17:56:26.540723 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:26.540684    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"]
Jul 17 17:56:26.542730 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:26.542707    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-flkk4"]
Jul 17 17:56:28.515541 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:28.515473    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" path="/var/lib/kubelet/pods/98c75cd2-db42-43fa-8791-5ffafde80558/volumes"
Jul 17 17:56:34.149532 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:34.149480    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:35.798503 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:35.798433    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:42.628328 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:42.628237    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:42.628328 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:42.628279    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:44.158888 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:44.158848    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fff5c4_f544_4e45_8a73_f852ca5d55e8.slice/crio-conmon-bc676d925483e8dc546830d75e877d5bc8ac1cc5dd0bf5ab295c44d6e598f603.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 17:56:47.455436 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455402    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"]
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455751    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455764    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455773    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455780    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455792    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455797    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455805    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455811    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455818    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455824    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455836    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455841    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455849    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455854    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455860    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455865    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455871    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerName="revoke-keys"
Jul 17 17:56:47.455870 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455876    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerName="revoke-keys"
Jul 17 17:56:47.456442 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455926    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" containerName="revoke-keys"
Jul 17 17:56:47.456442 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455933    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerName="revoke-keys"
Jul 17 17:56:47.456442 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455939    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerName="revoke-keys"
Jul 17 17:56:47.456442 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455946    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerName="revoke-keys"
Jul 17 17:56:47.456442 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455952    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerName="revoke-keys"
Jul 17 17:56:47.456442 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455957    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" containerName="revoke-keys"
Jul 17 17:56:47.456442 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.455962    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c75cd2-db42-43fa-8791-5ffafde80558" containerName="revoke-keys"
Jul 17 17:56:47.460381 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.460361    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"
Jul 17 17:56:47.463185 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.463152    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:56:47.463326 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.463153    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:56:47.467685 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.467650    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"]
Jul 17 17:56:47.529619 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.529576    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d6ef8a-0d86-4b3e-a5aa-84062da32da0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv\" (UID: \"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"
Jul 17 17:56:47.631083 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.631050    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d6ef8a-0d86-4b3e-a5aa-84062da32da0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv\" (UID: \"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"
Jul 17 17:56:47.631800 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.631774    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d6ef8a-0d86-4b3e-a5aa-84062da32da0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv\" (UID: \"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"
Jul 17 17:56:47.772661 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.772555    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"
Jul 17 17:56:47.914394 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:47.914360    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"]
Jul 17 17:56:47.916596 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:56:47.916563    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d6ef8a_0d86_4b3e_a5aa_84062da32da0.slice/crio-e24b7aa1d6d49bd56a32420f791ed5cadbe26657d7986bd64872f8f8f3c29df5 WatchSource:0}: Error finding container e24b7aa1d6d49bd56a32420f791ed5cadbe26657d7986bd64872f8f8f3c29df5: Status 404 returned error can't find the container with id e24b7aa1d6d49bd56a32420f791ed5cadbe26657d7986bd64872f8f8f3c29df5
Jul 17 17:56:48.635462 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:48.635424    2572 generic.go:356] "Generic (PLEG): container finished" podID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerID="d1a40e8cba63390087a5507df95a22176e6a3689bf6b64e4fb2807e0dbcf521f" exitCode=6
Jul 17 17:56:48.635898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:48.635522    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv" event={"ID":"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0","Type":"ContainerDied","Data":"d1a40e8cba63390087a5507df95a22176e6a3689bf6b64e4fb2807e0dbcf521f"}
Jul 17 17:56:48.635898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:48.635581    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv" event={"ID":"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0","Type":"ContainerStarted","Data":"e24b7aa1d6d49bd56a32420f791ed5cadbe26657d7986bd64872f8f8f3c29df5"}
Jul 17 17:56:48.635898 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:48.635710    2572 scope.go:117] "RemoveContainer" containerID="d1a40e8cba63390087a5507df95a22176e6a3689bf6b64e4fb2807e0dbcf521f"
Jul 17 17:56:48.976519 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:48.976464    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"]
Jul 17 17:56:48.976894 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:48.976880    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="93fff5c4-f544-4e45-8a73-f852ca5d55e8" containerName="revoke-keys"
Jul 17 17:56:48.976942 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:48.976897    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a86243d-a540-4ba9-b22d-ea38ff8a3fcd" containerName="revoke-keys"
Jul 17 17:56:48.980026 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:48.980007    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"
Jul 17 17:56:48.989363 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:48.989335    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"]
Jul 17 17:56:49.042836 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:49.042809    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efce05c6-98fc-4a31-9648-2f4061df4e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-h7fng\" (UID: \"efce05c6-98fc-4a31-9648-2f4061df4e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"
Jul 17 17:56:49.143659 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:49.143552    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efce05c6-98fc-4a31-9648-2f4061df4e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-h7fng\" (UID: \"efce05c6-98fc-4a31-9648-2f4061df4e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"
Jul 17 17:56:49.144207 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:49.144185    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efce05c6-98fc-4a31-9648-2f4061df4e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-h7fng\" (UID: \"efce05c6-98fc-4a31-9648-2f4061df4e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"
Jul 17 17:56:49.292041 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:49.292007    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"
Jul 17 17:56:49.643043 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:49.642964    2572 generic.go:356] "Generic (PLEG): container finished" podID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerID="fefcfd3da53f5801019a85e6381593cd70386d43a264a2cfd6d17c3db8be11a3" exitCode=6
Jul 17 17:56:49.643755 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:49.643375    2572 scope.go:117] "RemoveContainer" containerID="fefcfd3da53f5801019a85e6381593cd70386d43a264a2cfd6d17c3db8be11a3"
Jul 17 17:56:49.643755 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:49.643442    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv" event={"ID":"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0","Type":"ContainerDied","Data":"fefcfd3da53f5801019a85e6381593cd70386d43a264a2cfd6d17c3db8be11a3"}
Jul 17 17:56:49.643755 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:49.643578    2572 scope.go:117] "RemoveContainer" containerID="d1a40e8cba63390087a5507df95a22176e6a3689bf6b64e4fb2807e0dbcf521f"
Jul 17 17:56:49.643755 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:49.643641    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv_odh-ai-gateway-infra(f3d6ef8a-0d86-4b3e-a5aa-84062da32da0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0"
Jul 17 17:56:49.647634 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:49.647217    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"]
Jul 17 17:56:50.650627 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:50.650594    2572 scope.go:117] "RemoveContainer" containerID="fefcfd3da53f5801019a85e6381593cd70386d43a264a2cfd6d17c3db8be11a3"
Jul 17 17:56:50.651058 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:50.650860    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv_odh-ai-gateway-infra(f3d6ef8a-0d86-4b3e-a5aa-84062da32da0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0"
Jul 17 17:56:50.652000 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:50.651969    2572 generic.go:356] "Generic (PLEG): container finished" podID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerID="9ed1f1ca7ec37e036ce04b418d662d2f638c4e47d5863abd34e0e945b73efd03" exitCode=6
Jul 17 17:56:50.652122 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:50.652012    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng" event={"ID":"efce05c6-98fc-4a31-9648-2f4061df4e5a","Type":"ContainerDied","Data":"9ed1f1ca7ec37e036ce04b418d662d2f638c4e47d5863abd34e0e945b73efd03"}
Jul 17 17:56:50.652122 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:50.652052    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng" event={"ID":"efce05c6-98fc-4a31-9648-2f4061df4e5a","Type":"ContainerStarted","Data":"31413c491d869e1b524f4c85229c670f74c78209bdf73c99bbf510c43cff3501"}
Jul 17 17:56:50.652218 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:50.652207    2572 scope.go:117] "RemoveContainer" containerID="9ed1f1ca7ec37e036ce04b418d662d2f638c4e47d5863abd34e0e945b73efd03"
Jul 17 17:56:51.659642 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:51.659543    2572 generic.go:356] "Generic (PLEG): container finished" podID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerID="aca01aefa276bb8847313430b0cd8ee907c02308363f1dda299d14f040682456" exitCode=6
Jul 17 17:56:51.660122 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:51.659636    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng" event={"ID":"efce05c6-98fc-4a31-9648-2f4061df4e5a","Type":"ContainerDied","Data":"aca01aefa276bb8847313430b0cd8ee907c02308363f1dda299d14f040682456"}
Jul 17 17:56:51.660122 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:51.659744    2572 scope.go:117] "RemoveContainer" containerID="9ed1f1ca7ec37e036ce04b418d662d2f638c4e47d5863abd34e0e945b73efd03"
Jul 17 17:56:51.660122 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:51.659849    2572 scope.go:117] "RemoveContainer" containerID="aca01aefa276bb8847313430b0cd8ee907c02308363f1dda299d14f040682456"
Jul 17 17:56:51.660122 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:51.660092    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-h7fng_odh-ai-gateway-infra(efce05c6-98fc-4a31-9648-2f4061df4e5a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a"
Jul 17 17:56:52.666663 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:56:52.666635    2572 scope.go:117] "RemoveContainer" containerID="aca01aefa276bb8847313430b0cd8ee907c02308363f1dda299d14f040682456"
Jul 17 17:56:52.667048 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:56:52.666858    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-h7fng_odh-ai-gateway-infra(efce05c6-98fc-4a31-9648-2f4061df4e5a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a"
Jul 17 17:57:01.509840 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:01.509794    2572 scope.go:117] "RemoveContainer" containerID="fefcfd3da53f5801019a85e6381593cd70386d43a264a2cfd6d17c3db8be11a3"
Jul 17 17:57:02.719806 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:02.719771    2572 generic.go:356] "Generic (PLEG): container finished" podID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerID="cb4abff650c0ee4cf52e3351639dacea2d01d88e682333527bb80e620dd36adc" exitCode=6
Jul 17 17:57:02.720296 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:02.719841    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv" event={"ID":"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0","Type":"ContainerDied","Data":"cb4abff650c0ee4cf52e3351639dacea2d01d88e682333527bb80e620dd36adc"}
Jul 17 17:57:02.720296 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:02.719912    2572 scope.go:117] "RemoveContainer" containerID="fefcfd3da53f5801019a85e6381593cd70386d43a264a2cfd6d17c3db8be11a3"
Jul 17 17:57:02.720296 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:02.720251    2572 scope.go:117] "RemoveContainer" containerID="cb4abff650c0ee4cf52e3351639dacea2d01d88e682333527bb80e620dd36adc"
Jul 17 17:57:02.720468 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:02.720447    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv_odh-ai-gateway-infra(f3d6ef8a-0d86-4b3e-a5aa-84062da32da0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0"
Jul 17 17:57:03.751255 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:03.751215    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"]
Jul 17 17:57:03.896392 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:03.896365    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"
Jul 17 17:57:03.993444 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:03.993399    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d6ef8a-0d86-4b3e-a5aa-84062da32da0-maas-api-service-ca\") pod \"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0\" (UID: \"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0\") "
Jul 17 17:57:03.993814 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:03.993789    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3d6ef8a-0d86-4b3e-a5aa-84062da32da0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" (UID: "f3d6ef8a-0d86-4b3e-a5aa-84062da32da0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:57:04.094885 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:04.094837    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d6ef8a-0d86-4b3e-a5aa-84062da32da0-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:57:04.509969 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:04.509883    2572 scope.go:117] "RemoveContainer" containerID="aca01aefa276bb8847313430b0cd8ee907c02308363f1dda299d14f040682456"
Jul 17 17:57:04.735225 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:04.735189    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"
Jul 17 17:57:04.735225 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:04.735205    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv" event={"ID":"f3d6ef8a-0d86-4b3e-a5aa-84062da32da0","Type":"ContainerDied","Data":"e24b7aa1d6d49bd56a32420f791ed5cadbe26657d7986bd64872f8f8f3c29df5"}
Jul 17 17:57:04.735506 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:04.735276    2572 scope.go:117] "RemoveContainer" containerID="cb4abff650c0ee4cf52e3351639dacea2d01d88e682333527bb80e620dd36adc"
Jul 17 17:57:04.759875 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:04.759841    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"]
Jul 17 17:57:04.766295 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:04.766209    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-jz9gv"]
Jul 17 17:57:05.742618 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.742584    2572 generic.go:356] "Generic (PLEG): container finished" podID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerID="1d29904a9796217094217b3d7c234b17b401b7b8e8fdff05df555d6cfc3c2297" exitCode=6
Jul 17 17:57:05.742797 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.742663    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng" event={"ID":"efce05c6-98fc-4a31-9648-2f4061df4e5a","Type":"ContainerDied","Data":"1d29904a9796217094217b3d7c234b17b401b7b8e8fdff05df555d6cfc3c2297"}
Jul 17 17:57:05.742797 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.742730    2572 scope.go:117] "RemoveContainer" containerID="aca01aefa276bb8847313430b0cd8ee907c02308363f1dda299d14f040682456"
Jul 17 17:57:05.743058 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.743041    2572 scope.go:117] "RemoveContainer" containerID="1d29904a9796217094217b3d7c234b17b401b7b8e8fdff05df555d6cfc3c2297"
Jul 17 17:57:05.743341 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:05.743311    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-h7fng_odh-ai-gateway-infra(efce05c6-98fc-4a31-9648-2f4061df4e5a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a"
Jul 17 17:57:05.928333 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.928298    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"]
Jul 17 17:57:05.928781 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.928661    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerName="revoke-keys"
Jul 17 17:57:05.928781 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.928674    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerName="revoke-keys"
Jul 17 17:57:05.928781 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.928696    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerName="revoke-keys"
Jul 17 17:57:05.928781 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.928703    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerName="revoke-keys"
Jul 17 17:57:05.928781 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.928757    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerName="revoke-keys"
Jul 17 17:57:05.928781 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.928767    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerName="revoke-keys"
Jul 17 17:57:05.933010 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.932990    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"
Jul 17 17:57:05.943798 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:05.943762    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"]
Jul 17 17:57:06.009321 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.009227    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595cb3ec-bee5-4428-bd2f-0e1bd9de8362-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-x4b69\" (UID: \"595cb3ec-bee5-4428-bd2f-0e1bd9de8362\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"
Jul 17 17:57:06.109855 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.109813    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595cb3ec-bee5-4428-bd2f-0e1bd9de8362-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-x4b69\" (UID: \"595cb3ec-bee5-4428-bd2f-0e1bd9de8362\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"
Jul 17 17:57:06.110439 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.110418    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595cb3ec-bee5-4428-bd2f-0e1bd9de8362-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-x4b69\" (UID: \"595cb3ec-bee5-4428-bd2f-0e1bd9de8362\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"
Jul 17 17:57:06.246362 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.246320    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"
Jul 17 17:57:06.386267 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.386241    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"]
Jul 17 17:57:06.388576 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:57:06.388548    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595cb3ec_bee5_4428_bd2f_0e1bd9de8362.slice/crio-6a9e0141db2300ac675f20fb7124c9359f5820df6c343fbce9865ba097af77e6 WatchSource:0}: Error finding container 6a9e0141db2300ac675f20fb7124c9359f5820df6c343fbce9865ba097af77e6: Status 404 returned error can't find the container with id 6a9e0141db2300ac675f20fb7124c9359f5820df6c343fbce9865ba097af77e6
Jul 17 17:57:06.516057 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.516021    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" path="/var/lib/kubelet/pods/f3d6ef8a-0d86-4b3e-a5aa-84062da32da0/volumes"
Jul 17 17:57:06.752048 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.752009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" event={"ID":"595cb3ec-bee5-4428-bd2f-0e1bd9de8362","Type":"ContainerStarted","Data":"6c925ab0be5b6badf6870cb3213bd42c861c4b76e27da8af0f5428a729f9f81e"}
Jul 17 17:57:06.752265 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.752072    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" event={"ID":"595cb3ec-bee5-4428-bd2f-0e1bd9de8362","Type":"ContainerStarted","Data":"6a9e0141db2300ac675f20fb7124c9359f5820df6c343fbce9865ba097af77e6"}
Jul 17 17:57:06.771322 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.771263    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" podStartSLOduration=1.771246349 podStartE2EDuration="1.771246349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:06.770285692 +0000 UTC m=+1998.775641546" watchObservedRunningTime="2026-07-17 17:57:06.771246349 +0000 UTC m=+1998.776602203"
Jul 17 17:57:06.777235 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.777196    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"]
Jul 17 17:57:06.915739 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:06.915712    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"
Jul 17 17:57:07.016886 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.016849    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efce05c6-98fc-4a31-9648-2f4061df4e5a-maas-api-service-ca\") pod \"efce05c6-98fc-4a31-9648-2f4061df4e5a\" (UID: \"efce05c6-98fc-4a31-9648-2f4061df4e5a\") "
Jul 17 17:57:07.017292 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.017191    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efce05c6-98fc-4a31-9648-2f4061df4e5a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "efce05c6-98fc-4a31-9648-2f4061df4e5a" (UID: "efce05c6-98fc-4a31-9648-2f4061df4e5a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:57:07.117625 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.117595    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efce05c6-98fc-4a31-9648-2f4061df4e5a-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:57:07.758589 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.758544    2572 generic.go:356] "Generic (PLEG): container finished" podID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerID="6c925ab0be5b6badf6870cb3213bd42c861c4b76e27da8af0f5428a729f9f81e" exitCode=7
Jul 17 17:57:07.758787 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.758617    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" event={"ID":"595cb3ec-bee5-4428-bd2f-0e1bd9de8362","Type":"ContainerDied","Data":"6c925ab0be5b6badf6870cb3213bd42c861c4b76e27da8af0f5428a729f9f81e"}
Jul 17 17:57:07.758922 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.758906    2572 scope.go:117] "RemoveContainer" containerID="6c925ab0be5b6badf6870cb3213bd42c861c4b76e27da8af0f5428a729f9f81e"
Jul 17 17:57:07.760146 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.760121    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng" event={"ID":"efce05c6-98fc-4a31-9648-2f4061df4e5a","Type":"ContainerDied","Data":"31413c491d869e1b524f4c85229c670f74c78209bdf73c99bbf510c43cff3501"}
Jul 17 17:57:07.760249 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.760141    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"
Jul 17 17:57:07.760249 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.760176    2572 scope.go:117] "RemoveContainer" containerID="1d29904a9796217094217b3d7c234b17b401b7b8e8fdff05df555d6cfc3c2297"
Jul 17 17:57:07.798618 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.798581    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"]
Jul 17 17:57:07.800788 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:07.800764    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-h7fng"]
Jul 17 17:57:08.516246 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:08.516210    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" path="/var/lib/kubelet/pods/efce05c6-98fc-4a31-9648-2f4061df4e5a/volumes"
Jul 17 17:57:08.767997 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:08.767907    2572 generic.go:356] "Generic (PLEG): container finished" podID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerID="6883f4e5649467504362e6f872c339333a30112351b06ae839dc2528edf6ce88" exitCode=7
Jul 17 17:57:08.768166 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:08.767994    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" event={"ID":"595cb3ec-bee5-4428-bd2f-0e1bd9de8362","Type":"ContainerDied","Data":"6883f4e5649467504362e6f872c339333a30112351b06ae839dc2528edf6ce88"}
Jul 17 17:57:08.768166 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:08.768053    2572 scope.go:117] "RemoveContainer" containerID="6c925ab0be5b6badf6870cb3213bd42c861c4b76e27da8af0f5428a729f9f81e"
Jul 17 17:57:08.768333 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:08.768316    2572 scope.go:117] "RemoveContainer" containerID="6883f4e5649467504362e6f872c339333a30112351b06ae839dc2528edf6ce88"
Jul 17 17:57:08.768560 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:08.768541    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-x4b69_odh-ai-gateway-infra(595cb3ec-bee5-4428-bd2f-0e1bd9de8362)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362"
Jul 17 17:57:09.775431 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:09.775397    2572 scope.go:117] "RemoveContainer" containerID="6883f4e5649467504362e6f872c339333a30112351b06ae839dc2528edf6ce88"
Jul 17 17:57:09.775881 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:09.775603    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-x4b69_odh-ai-gateway-infra(595cb3ec-bee5-4428-bd2f-0e1bd9de8362)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362"
Jul 17 17:57:21.509808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:21.509770    2572 scope.go:117] "RemoveContainer" containerID="6883f4e5649467504362e6f872c339333a30112351b06ae839dc2528edf6ce88"
Jul 17 17:57:22.847682 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:22.847644    2572 generic.go:356] "Generic (PLEG): container finished" podID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerID="cc20a909ab0f58a33e8da1c472d77ce5a6c77ac75d4644673229ee39706804d6" exitCode=7
Jul 17 17:57:22.848159 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:22.847712    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" event={"ID":"595cb3ec-bee5-4428-bd2f-0e1bd9de8362","Type":"ContainerDied","Data":"cc20a909ab0f58a33e8da1c472d77ce5a6c77ac75d4644673229ee39706804d6"}
Jul 17 17:57:22.848159 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:22.847784    2572 scope.go:117] "RemoveContainer" containerID="6883f4e5649467504362e6f872c339333a30112351b06ae839dc2528edf6ce88"
Jul 17 17:57:22.848159 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:22.848146    2572 scope.go:117] "RemoveContainer" containerID="cc20a909ab0f58a33e8da1c472d77ce5a6c77ac75d4644673229ee39706804d6"
Jul 17 17:57:22.848407 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:22.848382    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-x4b69_odh-ai-gateway-infra(595cb3ec-bee5-4428-bd2f-0e1bd9de8362)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362"
Jul 17 17:57:23.876444 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:23.876398    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"]
Jul 17 17:57:24.019439 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:24.019414    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"
Jul 17 17:57:24.073158 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:24.073113    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595cb3ec-bee5-4428-bd2f-0e1bd9de8362-maas-api-service-ca\") pod \"595cb3ec-bee5-4428-bd2f-0e1bd9de8362\" (UID: \"595cb3ec-bee5-4428-bd2f-0e1bd9de8362\") "
Jul 17 17:57:24.073519 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:24.073471    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/595cb3ec-bee5-4428-bd2f-0e1bd9de8362-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "595cb3ec-bee5-4428-bd2f-0e1bd9de8362" (UID: "595cb3ec-bee5-4428-bd2f-0e1bd9de8362"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:57:24.174806 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:24.174705    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595cb3ec-bee5-4428-bd2f-0e1bd9de8362-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:57:24.861503 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:24.861459    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"
Jul 17 17:57:24.861697 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:24.861456    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69" event={"ID":"595cb3ec-bee5-4428-bd2f-0e1bd9de8362","Type":"ContainerDied","Data":"6a9e0141db2300ac675f20fb7124c9359f5820df6c343fbce9865ba097af77e6"}
Jul 17 17:57:24.861697 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:24.861601    2572 scope.go:117] "RemoveContainer" containerID="cc20a909ab0f58a33e8da1c472d77ce5a6c77ac75d4644673229ee39706804d6"
Jul 17 17:57:24.901654 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:24.901619    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"]
Jul 17 17:57:24.914388 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:24.914353    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-x4b69"]
Jul 17 17:57:26.515065 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:26.515027    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" path="/var/lib/kubelet/pods/595cb3ec-bee5-4428-bd2f-0e1bd9de8362/volumes"
Jul 17 17:57:37.483454 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483409    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"]
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483806    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerName="revoke-keys"
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483817    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerName="revoke-keys"
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483833    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerName="revoke-keys"
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483838    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerName="revoke-keys"
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483845    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerName="revoke-keys"
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483850    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerName="revoke-keys"
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483859    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerName="revoke-keys"
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483865    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerName="revoke-keys"
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483879    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerName="revoke-keys"
Jul 17 17:57:37.483914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483884    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerName="revoke-keys"
Jul 17 17:57:37.484222 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483932    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerName="revoke-keys"
Jul 17 17:57:37.484222 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483940    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerName="revoke-keys"
Jul 17 17:57:37.484222 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483948    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerName="revoke-keys"
Jul 17 17:57:37.484222 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483964    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerName="revoke-keys"
Jul 17 17:57:37.484222 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483971    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerName="revoke-keys"
Jul 17 17:57:37.484222 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.483978    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d6ef8a-0d86-4b3e-a5aa-84062da32da0" containerName="revoke-keys"
Jul 17 17:57:37.488225 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.488204    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"
Jul 17 17:57:37.491287 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.491254    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:57:37.492560 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.492517    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:57:37.494719 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.494688    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"]
Jul 17 17:57:37.596115 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.596068    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ba45a5-b167-433d-a807-d3620d3eb1d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt\" (UID: \"c4ba45a5-b167-433d-a807-d3620d3eb1d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"
Jul 17 17:57:37.696730 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.696685    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ba45a5-b167-433d-a807-d3620d3eb1d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt\" (UID: \"c4ba45a5-b167-433d-a807-d3620d3eb1d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"
Jul 17 17:57:37.697320 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.697300    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ba45a5-b167-433d-a807-d3620d3eb1d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt\" (UID: \"c4ba45a5-b167-433d-a807-d3620d3eb1d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"
Jul 17 17:57:37.800658 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.800555    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"
Jul 17 17:57:37.951733 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:37.951678    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"]
Jul 17 17:57:37.954654 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:57:37.954615    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ba45a5_b167_433d_a807_d3620d3eb1d5.slice/crio-2be6b5778d3216a0f164758758928ab421e79129fde02b9a16629fb32b268e5a WatchSource:0}: Error finding container 2be6b5778d3216a0f164758758928ab421e79129fde02b9a16629fb32b268e5a: Status 404 returned error can't find the container with id 2be6b5778d3216a0f164758758928ab421e79129fde02b9a16629fb32b268e5a
Jul 17 17:57:38.937073 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:38.936965    2572 generic.go:356] "Generic (PLEG): container finished" podID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerID="e87db574c7faac6d1480958be6b4a5946bdecbaea5da482d13be5b3ec2f4c942" exitCode=6
Jul 17 17:57:38.937073 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:38.937003    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" event={"ID":"c4ba45a5-b167-433d-a807-d3620d3eb1d5","Type":"ContainerDied","Data":"e87db574c7faac6d1480958be6b4a5946bdecbaea5da482d13be5b3ec2f4c942"}
Jul 17 17:57:38.937073 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:38.937065    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" event={"ID":"c4ba45a5-b167-433d-a807-d3620d3eb1d5","Type":"ContainerStarted","Data":"2be6b5778d3216a0f164758758928ab421e79129fde02b9a16629fb32b268e5a"}
Jul 17 17:57:38.937631 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:38.937223    2572 scope.go:117] "RemoveContainer" containerID="e87db574c7faac6d1480958be6b4a5946bdecbaea5da482d13be5b3ec2f4c942"
Jul 17 17:57:39.024509 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.024464    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"]
Jul 17 17:57:39.024831 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.024817    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerName="revoke-keys"
Jul 17 17:57:39.024876 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.024833    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efce05c6-98fc-4a31-9648-2f4061df4e5a" containerName="revoke-keys"
Jul 17 17:57:39.024876 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.024850    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerName="revoke-keys"
Jul 17 17:57:39.024876 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.024856    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerName="revoke-keys"
Jul 17 17:57:39.024974 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.024914    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="595cb3ec-bee5-4428-bd2f-0e1bd9de8362" containerName="revoke-keys"
Jul 17 17:57:39.028914 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.028895    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"
Jul 17 17:57:39.040449 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.040417    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"]
Jul 17 17:57:39.109115 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.109068    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0d5b23-3704-4035-a9cb-ab0da60b2780-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-9td5w\" (UID: \"1a0d5b23-3704-4035-a9cb-ab0da60b2780\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"
Jul 17 17:57:39.209949 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.209850    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0d5b23-3704-4035-a9cb-ab0da60b2780-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-9td5w\" (UID: \"1a0d5b23-3704-4035-a9cb-ab0da60b2780\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"
Jul 17 17:57:39.210623 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.210603    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0d5b23-3704-4035-a9cb-ab0da60b2780-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-9td5w\" (UID: \"1a0d5b23-3704-4035-a9cb-ab0da60b2780\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"
Jul 17 17:57:39.339829 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.339786    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"
Jul 17 17:57:39.479585 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.479558    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"]
Jul 17 17:57:39.481750 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:57:39.481716    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0d5b23_3704_4035_a9cb_ab0da60b2780.slice/crio-f05a09c8c8462fb7d7f3d5aba0da6553b99e86b4572dd76f058895be126efcd9 WatchSource:0}: Error finding container f05a09c8c8462fb7d7f3d5aba0da6553b99e86b4572dd76f058895be126efcd9: Status 404 returned error can't find the container with id f05a09c8c8462fb7d7f3d5aba0da6553b99e86b4572dd76f058895be126efcd9
Jul 17 17:57:39.943790 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.943749    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" event={"ID":"1a0d5b23-3704-4035-a9cb-ab0da60b2780","Type":"ContainerStarted","Data":"2fe4347653bda8f9e883bebf3ac87012cf1e475c6700081d242f9387624744c3"}
Jul 17 17:57:39.944282 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.943808    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" event={"ID":"1a0d5b23-3704-4035-a9cb-ab0da60b2780","Type":"ContainerStarted","Data":"f05a09c8c8462fb7d7f3d5aba0da6553b99e86b4572dd76f058895be126efcd9"}
Jul 17 17:57:39.945698 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.945664    2572 generic.go:356] "Generic (PLEG): container finished" podID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerID="0afaa1b9a9fdf2508956de7f39b4bacf37b64683fab63f0bd5795a502ab0e4aa" exitCode=6
Jul 17 17:57:39.945848 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.945713    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" event={"ID":"c4ba45a5-b167-433d-a807-d3620d3eb1d5","Type":"ContainerDied","Data":"0afaa1b9a9fdf2508956de7f39b4bacf37b64683fab63f0bd5795a502ab0e4aa"}
Jul 17 17:57:39.945848 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.945766    2572 scope.go:117] "RemoveContainer" containerID="e87db574c7faac6d1480958be6b4a5946bdecbaea5da482d13be5b3ec2f4c942"
Jul 17 17:57:39.945956 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.945941    2572 scope.go:117] "RemoveContainer" containerID="0afaa1b9a9fdf2508956de7f39b4bacf37b64683fab63f0bd5795a502ab0e4aa"
Jul 17 17:57:39.946197 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:39.946175    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt_odh-ai-gateway-infra(c4ba45a5-b167-433d-a807-d3620d3eb1d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5"
Jul 17 17:57:39.964374 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:39.964329    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" podStartSLOduration=0.964311211 podStartE2EDuration="964.311211ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:39.961641349 +0000 UTC m=+2031.966997222" watchObservedRunningTime="2026-07-17 17:57:39.964311211 +0000 UTC m=+2031.969667065"
Jul 17 17:57:40.952855 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:40.952820    2572 scope.go:117] "RemoveContainer" containerID="0afaa1b9a9fdf2508956de7f39b4bacf37b64683fab63f0bd5795a502ab0e4aa"
Jul 17 17:57:40.953289 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:40.953081    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt_odh-ai-gateway-infra(c4ba45a5-b167-433d-a807-d3620d3eb1d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5"
Jul 17 17:57:40.954220 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:40.954192    2572 generic.go:356] "Generic (PLEG): container finished" podID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerID="2fe4347653bda8f9e883bebf3ac87012cf1e475c6700081d242f9387624744c3" exitCode=6
Jul 17 17:57:40.954355 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:40.954283    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" event={"ID":"1a0d5b23-3704-4035-a9cb-ab0da60b2780","Type":"ContainerDied","Data":"2fe4347653bda8f9e883bebf3ac87012cf1e475c6700081d242f9387624744c3"}
Jul 17 17:57:40.954443 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:40.954429    2572 scope.go:117] "RemoveContainer" containerID="2fe4347653bda8f9e883bebf3ac87012cf1e475c6700081d242f9387624744c3"
Jul 17 17:57:41.961633 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:41.961531    2572 generic.go:356] "Generic (PLEG): container finished" podID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerID="a55ad11f696cb2a0eda82b2f98f172eb83e5035ee4a5b82b73c7af7c524abf6f" exitCode=6
Jul 17 17:57:41.961633 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:41.961617    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" event={"ID":"1a0d5b23-3704-4035-a9cb-ab0da60b2780","Type":"ContainerDied","Data":"a55ad11f696cb2a0eda82b2f98f172eb83e5035ee4a5b82b73c7af7c524abf6f"}
Jul 17 17:57:41.962108 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:41.961682    2572 scope.go:117] "RemoveContainer" containerID="2fe4347653bda8f9e883bebf3ac87012cf1e475c6700081d242f9387624744c3"
Jul 17 17:57:41.962108 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:41.961865    2572 scope.go:117] "RemoveContainer" containerID="a55ad11f696cb2a0eda82b2f98f172eb83e5035ee4a5b82b73c7af7c524abf6f"
Jul 17 17:57:41.962108 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:41.962081    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-9td5w_odh-ai-gateway-infra(1a0d5b23-3704-4035-a9cb-ab0da60b2780)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780"
Jul 17 17:57:42.969713 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:42.969685    2572 scope.go:117] "RemoveContainer" containerID="a55ad11f696cb2a0eda82b2f98f172eb83e5035ee4a5b82b73c7af7c524abf6f"
Jul 17 17:57:42.970116 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:42.969902    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-9td5w_odh-ai-gateway-infra(1a0d5b23-3704-4035-a9cb-ab0da60b2780)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780"
Jul 17 17:57:53.509843 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:53.509807    2572 scope.go:117] "RemoveContainer" containerID="0afaa1b9a9fdf2508956de7f39b4bacf37b64683fab63f0bd5795a502ab0e4aa"
Jul 17 17:57:54.027607 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:54.027564    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" event={"ID":"c4ba45a5-b167-433d-a807-d3620d3eb1d5","Type":"ContainerStarted","Data":"6434cf940c47d515b07c45bc7492d1a35bf8c74f29bd97500825ef5d14f7aec2"}
Jul 17 17:57:54.045476 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:54.045417    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" podStartSLOduration=17.045401081 podStartE2EDuration="17.045401081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:54.043390343 +0000 UTC m=+2046.048746203" watchObservedRunningTime="2026-07-17 17:57:54.045401081 +0000 UTC m=+2046.050756935"
Jul 17 17:57:54.510269 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:54.510176    2572 scope.go:117] "RemoveContainer" containerID="a55ad11f696cb2a0eda82b2f98f172eb83e5035ee4a5b82b73c7af7c524abf6f"
Jul 17 17:57:55.035158 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:55.035122    2572 generic.go:356] "Generic (PLEG): container finished" podID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerID="6434cf940c47d515b07c45bc7492d1a35bf8c74f29bd97500825ef5d14f7aec2" exitCode=6
Jul 17 17:57:55.035158 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:55.035147    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" event={"ID":"c4ba45a5-b167-433d-a807-d3620d3eb1d5","Type":"ContainerDied","Data":"6434cf940c47d515b07c45bc7492d1a35bf8c74f29bd97500825ef5d14f7aec2"}
Jul 17 17:57:55.035455 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:55.035208    2572 scope.go:117] "RemoveContainer" containerID="0afaa1b9a9fdf2508956de7f39b4bacf37b64683fab63f0bd5795a502ab0e4aa"
Jul 17 17:57:55.035589 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:55.035562    2572 scope.go:117] "RemoveContainer" containerID="6434cf940c47d515b07c45bc7492d1a35bf8c74f29bd97500825ef5d14f7aec2"
Jul 17 17:57:55.035819 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:55.035798    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt_odh-ai-gateway-infra(c4ba45a5-b167-433d-a807-d3620d3eb1d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5"
Jul 17 17:57:55.037445 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:55.037422    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" event={"ID":"1a0d5b23-3704-4035-a9cb-ab0da60b2780","Type":"ContainerStarted","Data":"381ce7c15b480f8f1725df588a6b3636bc10a8e3f1c9a61a89f211d1ff7d56bc"}
Jul 17 17:57:55.060989 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:55.060954    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"]
Jul 17 17:57:55.971454 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:55.971415    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"]
Jul 17 17:57:55.974782 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:55.974763    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"
Jul 17 17:57:55.984432 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:55.984406    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"]
Jul 17 17:57:56.047583 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.047552    2572 generic.go:356] "Generic (PLEG): container finished" podID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerID="381ce7c15b480f8f1725df588a6b3636bc10a8e3f1c9a61a89f211d1ff7d56bc" exitCode=6
Jul 17 17:57:56.047713 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.047635    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" event={"ID":"1a0d5b23-3704-4035-a9cb-ab0da60b2780","Type":"ContainerDied","Data":"381ce7c15b480f8f1725df588a6b3636bc10a8e3f1c9a61a89f211d1ff7d56bc"}
Jul 17 17:57:56.047762 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.047713    2572 scope.go:117] "RemoveContainer" containerID="a55ad11f696cb2a0eda82b2f98f172eb83e5035ee4a5b82b73c7af7c524abf6f"
Jul 17 17:57:56.048029 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.048013    2572 scope.go:117] "RemoveContainer" containerID="381ce7c15b480f8f1725df588a6b3636bc10a8e3f1c9a61a89f211d1ff7d56bc"
Jul 17 17:57:56.048257 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:56.048239    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-9td5w_odh-ai-gateway-infra(1a0d5b23-3704-4035-a9cb-ab0da60b2780)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780"
Jul 17 17:57:56.056330 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.056304    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafede7-1743-42ae-a5c6-9ce0cfbe74a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp\" (UID: \"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"
Jul 17 17:57:56.079449 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.079413    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"]
Jul 17 17:57:56.157827 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.157779    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafede7-1743-42ae-a5c6-9ce0cfbe74a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp\" (UID: \"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"
Jul 17 17:57:56.158475 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.158443    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafede7-1743-42ae-a5c6-9ce0cfbe74a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp\" (UID: \"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"
Jul 17 17:57:56.181499 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.181457    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"
Jul 17 17:57:56.258731 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.258630    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ba45a5-b167-433d-a807-d3620d3eb1d5-maas-api-service-ca\") pod \"c4ba45a5-b167-433d-a807-d3620d3eb1d5\" (UID: \"c4ba45a5-b167-433d-a807-d3620d3eb1d5\") "
Jul 17 17:57:56.258983 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.258958    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ba45a5-b167-433d-a807-d3620d3eb1d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c4ba45a5-b167-433d-a807-d3620d3eb1d5" (UID: "c4ba45a5-b167-433d-a807-d3620d3eb1d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:57:56.285537 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.285479    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"
Jul 17 17:57:56.360070 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.360036    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ba45a5-b167-433d-a807-d3620d3eb1d5-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:57:56.428792 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:56.428764    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"]
Jul 17 17:57:56.430608 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:57:56.430583    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bafede7_1743_42ae_a5c6_9ce0cfbe74a3.slice/crio-b503ca6ad5d6900340272fbf7d96f4b2e5d4864f54c82c49eecb87be98253a12 WatchSource:0}: Error finding container b503ca6ad5d6900340272fbf7d96f4b2e5d4864f54c82c49eecb87be98253a12: Status 404 returned error can't find the container with id b503ca6ad5d6900340272fbf7d96f4b2e5d4864f54c82c49eecb87be98253a12
Jul 17 17:57:57.053945 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.053915    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"
Jul 17 17:57:57.054427 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.053916    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt" event={"ID":"c4ba45a5-b167-433d-a807-d3620d3eb1d5","Type":"ContainerDied","Data":"2be6b5778d3216a0f164758758928ab421e79129fde02b9a16629fb32b268e5a"}
Jul 17 17:57:57.054427 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.054062    2572 scope.go:117] "RemoveContainer" containerID="6434cf940c47d515b07c45bc7492d1a35bf8c74f29bd97500825ef5d14f7aec2"
Jul 17 17:57:57.055616 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.055586    2572 generic.go:356] "Generic (PLEG): container finished" podID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerID="1e2e4a877c7ca49401ecc2c198fd88bb6cc4609426b17bf089b19aeb2c858fa5" exitCode=7
Jul 17 17:57:57.055764 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.055672    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp" event={"ID":"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3","Type":"ContainerDied","Data":"1e2e4a877c7ca49401ecc2c198fd88bb6cc4609426b17bf089b19aeb2c858fa5"}
Jul 17 17:57:57.055764 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.055722    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp" event={"ID":"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3","Type":"ContainerStarted","Data":"b503ca6ad5d6900340272fbf7d96f4b2e5d4864f54c82c49eecb87be98253a12"}
Jul 17 17:57:57.055888 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.055864    2572 scope.go:117] "RemoveContainer" containerID="1e2e4a877c7ca49401ecc2c198fd88bb6cc4609426b17bf089b19aeb2c858fa5"
Jul 17 17:57:57.112530 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.112471    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"]
Jul 17 17:57:57.118109 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.118093    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-f4tjt"]
Jul 17 17:57:57.211643 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.211611    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"
Jul 17 17:57:57.266920 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.266886    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0d5b23-3704-4035-a9cb-ab0da60b2780-maas-api-service-ca\") pod \"1a0d5b23-3704-4035-a9cb-ab0da60b2780\" (UID: \"1a0d5b23-3704-4035-a9cb-ab0da60b2780\") "
Jul 17 17:57:57.267276 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.267248    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a0d5b23-3704-4035-a9cb-ab0da60b2780-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a0d5b23-3704-4035-a9cb-ab0da60b2780" (UID: "1a0d5b23-3704-4035-a9cb-ab0da60b2780"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:57:57.368247 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:57.368200    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0d5b23-3704-4035-a9cb-ab0da60b2780-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:57:58.065394 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.065359    2572 generic.go:356] "Generic (PLEG): container finished" podID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerID="b47bb84e0dd0fcf69a185bb87069b8677b8c79f8640405035c38e0410318e890" exitCode=7
Jul 17 17:57:58.065891 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.065449    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp" event={"ID":"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3","Type":"ContainerDied","Data":"b47bb84e0dd0fcf69a185bb87069b8677b8c79f8640405035c38e0410318e890"}
Jul 17 17:57:58.065891 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.065529    2572 scope.go:117] "RemoveContainer" containerID="1e2e4a877c7ca49401ecc2c198fd88bb6cc4609426b17bf089b19aeb2c858fa5"
Jul 17 17:57:58.065891 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.065831    2572 scope.go:117] "RemoveContainer" containerID="b47bb84e0dd0fcf69a185bb87069b8677b8c79f8640405035c38e0410318e890"
Jul 17 17:57:58.066107 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:58.066081    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp_odh-ai-gateway-infra(8bafede7-1743-42ae-a5c6-9ce0cfbe74a3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3"
Jul 17 17:57:58.066978 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.066829    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"
Jul 17 17:57:58.066978 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.066839    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w" event={"ID":"1a0d5b23-3704-4035-a9cb-ab0da60b2780","Type":"ContainerDied","Data":"f05a09c8c8462fb7d7f3d5aba0da6553b99e86b4572dd76f058895be126efcd9"}
Jul 17 17:57:58.077621 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.077600    2572 scope.go:117] "RemoveContainer" containerID="381ce7c15b480f8f1725df588a6b3636bc10a8e3f1c9a61a89f211d1ff7d56bc"
Jul 17 17:57:58.098909 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.098829    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"]
Jul 17 17:57:58.101838 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.101809    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-9td5w"]
Jul 17 17:57:58.515640 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.515561    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" path="/var/lib/kubelet/pods/1a0d5b23-3704-4035-a9cb-ab0da60b2780/volumes"
Jul 17 17:57:58.515934 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:58.515921    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" path="/var/lib/kubelet/pods/c4ba45a5-b167-433d-a807-d3620d3eb1d5/volumes"
Jul 17 17:57:59.074102 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:57:59.074071    2572 scope.go:117] "RemoveContainer" containerID="b47bb84e0dd0fcf69a185bb87069b8677b8c79f8640405035c38e0410318e890"
Jul 17 17:57:59.074627 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:57:59.074324    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp_odh-ai-gateway-infra(8bafede7-1743-42ae-a5c6-9ce0cfbe74a3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3"
Jul 17 17:58:12.510178 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:12.510091    2572 scope.go:117] "RemoveContainer" containerID="b47bb84e0dd0fcf69a185bb87069b8677b8c79f8640405035c38e0410318e890"
Jul 17 17:58:13.153430 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:13.153396    2572 generic.go:356] "Generic (PLEG): container finished" podID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerID="f73dd46e51153b30e2064907e50209878eddf2bd559bfacc71da357609ee5059" exitCode=7
Jul 17 17:58:13.153626 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:13.153474    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp" event={"ID":"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3","Type":"ContainerDied","Data":"f73dd46e51153b30e2064907e50209878eddf2bd559bfacc71da357609ee5059"}
Jul 17 17:58:13.153626 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:13.153558    2572 scope.go:117] "RemoveContainer" containerID="b47bb84e0dd0fcf69a185bb87069b8677b8c79f8640405035c38e0410318e890"
Jul 17 17:58:13.153802 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:13.153788    2572 scope.go:117] "RemoveContainer" containerID="f73dd46e51153b30e2064907e50209878eddf2bd559bfacc71da357609ee5059"
Jul 17 17:58:13.154041 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:58:13.154020    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp_odh-ai-gateway-infra(8bafede7-1743-42ae-a5c6-9ce0cfbe74a3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3"
Jul 17 17:58:14.182772 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:14.182733    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"]
Jul 17 17:58:14.329715 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:14.329684    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"
Jul 17 17:58:14.411280 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:14.411231    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafede7-1743-42ae-a5c6-9ce0cfbe74a3-maas-api-service-ca\") pod \"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3\" (UID: \"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3\") "
Jul 17 17:58:14.411608 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:14.411574    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bafede7-1743-42ae-a5c6-9ce0cfbe74a3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" (UID: "8bafede7-1743-42ae-a5c6-9ce0cfbe74a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:58:14.511985 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:14.511893    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bafede7-1743-42ae-a5c6-9ce0cfbe74a3-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:58:15.167892 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:15.167860    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"
Jul 17 17:58:15.167892 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:15.167885    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp" event={"ID":"8bafede7-1743-42ae-a5c6-9ce0cfbe74a3","Type":"ContainerDied","Data":"b503ca6ad5d6900340272fbf7d96f4b2e5d4864f54c82c49eecb87be98253a12"}
Jul 17 17:58:15.168170 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:15.167953    2572 scope.go:117] "RemoveContainer" containerID="f73dd46e51153b30e2064907e50209878eddf2bd559bfacc71da357609ee5059"
Jul 17 17:58:15.191169 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:15.191126    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"]
Jul 17 17:58:15.197460 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:15.197426    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnxbp"]
Jul 17 17:58:16.516515 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:16.516452    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" path="/var/lib/kubelet/pods/8bafede7-1743-42ae-a5c6-9ce0cfbe74a3/volumes"
Jul 17 17:58:27.539089 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539052    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"]
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539380    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539391    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539397    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539403    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539414    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539420    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539427    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539432    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539442    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539449    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539456    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539461    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539472    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539477    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539501    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539507    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539564    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539571    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539577    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539582    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539588    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerName="revoke-keys"
Jul 17 17:58:27.539612 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.539595    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerName="revoke-keys"
Jul 17 17:58:27.542653 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.542633    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"
Jul 17 17:58:27.545811 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.545784    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:58:27.545971 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.545849    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:58:27.551730 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.551690    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"]
Jul 17 17:58:27.628215 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.628172    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6a5841-7be6-43ff-a37d-f3341997277a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q\" (UID: \"fa6a5841-7be6-43ff-a37d-f3341997277a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"
Jul 17 17:58:27.728794 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.728751    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6a5841-7be6-43ff-a37d-f3341997277a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q\" (UID: \"fa6a5841-7be6-43ff-a37d-f3341997277a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"
Jul 17 17:58:27.729452 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.729423    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6a5841-7be6-43ff-a37d-f3341997277a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q\" (UID: \"fa6a5841-7be6-43ff-a37d-f3341997277a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"
Jul 17 17:58:27.853933 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.853905    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"
Jul 17 17:58:27.996063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:27.996035    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"]
Jul 17 17:58:27.997961 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:58:27.997934    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa6a5841_7be6_43ff_a37d_f3341997277a.slice/crio-5faaa2923a0f742c93c2d01e1422a8f4cd5b20981ff7fe864a5b1ab7d52a017c WatchSource:0}: Error finding container 5faaa2923a0f742c93c2d01e1422a8f4cd5b20981ff7fe864a5b1ab7d52a017c: Status 404 returned error can't find the container with id 5faaa2923a0f742c93c2d01e1422a8f4cd5b20981ff7fe864a5b1ab7d52a017c
Jul 17 17:58:28.241012 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:28.240979    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q" event={"ID":"fa6a5841-7be6-43ff-a37d-f3341997277a","Type":"ContainerStarted","Data":"5faaa2923a0f742c93c2d01e1422a8f4cd5b20981ff7fe864a5b1ab7d52a017c"}
Jul 17 17:58:29.077928 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.077894    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"]
Jul 17 17:58:29.078331 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.078240    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerName="revoke-keys"
Jul 17 17:58:29.078331 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.078251    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerName="revoke-keys"
Jul 17 17:58:29.078331 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.078301    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bafede7-1743-42ae-a5c6-9ce0cfbe74a3" containerName="revoke-keys"
Jul 17 17:58:29.078331 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.078312    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ba45a5-b167-433d-a807-d3620d3eb1d5" containerName="revoke-keys"
Jul 17 17:58:29.078331 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.078320    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a0d5b23-3704-4035-a9cb-ab0da60b2780" containerName="revoke-keys"
Jul 17 17:58:29.081208 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.081190    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"
Jul 17 17:58:29.091959 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.091929    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"]
Jul 17 17:58:29.154128 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.154087    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b573986-98f8-46cf-8b19-cd2f6ec417d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd\" (UID: \"3b573986-98f8-46cf-8b19-cd2f6ec417d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"
Jul 17 17:58:29.248550 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.248473    2572 generic.go:356] "Generic (PLEG): container finished" podID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerID="e022d84e6b6fc09a7c2a2785b8214b38fc7b12cd638ed03bd1e6fd11292c4793" exitCode=6
Jul 17 17:58:29.248743 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.248579    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q" event={"ID":"fa6a5841-7be6-43ff-a37d-f3341997277a","Type":"ContainerDied","Data":"e022d84e6b6fc09a7c2a2785b8214b38fc7b12cd638ed03bd1e6fd11292c4793"}
Jul 17 17:58:29.248791 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.248766    2572 scope.go:117] "RemoveContainer" containerID="e022d84e6b6fc09a7c2a2785b8214b38fc7b12cd638ed03bd1e6fd11292c4793"
Jul 17 17:58:29.254835 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.254807    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b573986-98f8-46cf-8b19-cd2f6ec417d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd\" (UID: \"3b573986-98f8-46cf-8b19-cd2f6ec417d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"
Jul 17 17:58:29.255419 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.255380    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b573986-98f8-46cf-8b19-cd2f6ec417d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd\" (UID: \"3b573986-98f8-46cf-8b19-cd2f6ec417d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"
Jul 17 17:58:29.393128 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.393017    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"
Jul 17 17:58:29.534902 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:29.534869    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"]
Jul 17 17:58:29.537051 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:58:29.537009    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b573986_98f8_46cf_8b19_cd2f6ec417d8.slice/crio-18895e9886fce947662c9cc8d9bd564ade041d8b0908de7441e2fa44dfa02bae WatchSource:0}: Error finding container 18895e9886fce947662c9cc8d9bd564ade041d8b0908de7441e2fa44dfa02bae: Status 404 returned error can't find the container with id 18895e9886fce947662c9cc8d9bd564ade041d8b0908de7441e2fa44dfa02bae
Jul 17 17:58:30.255507 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:30.255464    2572 generic.go:356] "Generic (PLEG): container finished" podID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerID="51597a7cc30dfcbc3a950f1f31044acc74f9cbe7955923fcd35e8f024b9c0088" exitCode=6
Jul 17 17:58:30.255991 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:30.255544    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd" event={"ID":"3b573986-98f8-46cf-8b19-cd2f6ec417d8","Type":"ContainerDied","Data":"51597a7cc30dfcbc3a950f1f31044acc74f9cbe7955923fcd35e8f024b9c0088"}
Jul 17 17:58:30.255991 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:30.255599    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd" event={"ID":"3b573986-98f8-46cf-8b19-cd2f6ec417d8","Type":"ContainerStarted","Data":"18895e9886fce947662c9cc8d9bd564ade041d8b0908de7441e2fa44dfa02bae"}
Jul 17 17:58:30.255991 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:30.255895    2572 scope.go:117] "RemoveContainer" containerID="51597a7cc30dfcbc3a950f1f31044acc74f9cbe7955923fcd35e8f024b9c0088"
Jul 17 17:58:30.257662 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:30.257641    2572 generic.go:356] "Generic (PLEG): container finished" podID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerID="0d4ea1911424c46d144bab3eed2f3294840f1bf688818e884ccce391f69b590b" exitCode=6
Jul 17 17:58:30.257758 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:30.257677    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q" event={"ID":"fa6a5841-7be6-43ff-a37d-f3341997277a","Type":"ContainerDied","Data":"0d4ea1911424c46d144bab3eed2f3294840f1bf688818e884ccce391f69b590b"}
Jul 17 17:58:30.257758 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:30.257722    2572 scope.go:117] "RemoveContainer" containerID="e022d84e6b6fc09a7c2a2785b8214b38fc7b12cd638ed03bd1e6fd11292c4793"
Jul 17 17:58:30.257936 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:30.257920    2572 scope.go:117] "RemoveContainer" containerID="0d4ea1911424c46d144bab3eed2f3294840f1bf688818e884ccce391f69b590b"
Jul 17 17:58:30.258203 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:58:30.258130    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q_odh-ai-gateway-infra(fa6a5841-7be6-43ff-a37d-f3341997277a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a"
Jul 17 17:58:31.265154 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:31.265058    2572 generic.go:356] "Generic (PLEG): container finished" podID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerID="7112f53f17ff7258c5c611cacb0497c7f8dd50af09b0266e80a532acdd1eac28" exitCode=6
Jul 17 17:58:31.265658 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:31.265146    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd" event={"ID":"3b573986-98f8-46cf-8b19-cd2f6ec417d8","Type":"ContainerDied","Data":"7112f53f17ff7258c5c611cacb0497c7f8dd50af09b0266e80a532acdd1eac28"}
Jul 17 17:58:31.265658 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:31.265209    2572 scope.go:117] "RemoveContainer" containerID="51597a7cc30dfcbc3a950f1f31044acc74f9cbe7955923fcd35e8f024b9c0088"
Jul 17 17:58:31.265658 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:31.265460    2572 scope.go:117] "RemoveContainer" containerID="7112f53f17ff7258c5c611cacb0497c7f8dd50af09b0266e80a532acdd1eac28"
Jul 17 17:58:31.265818 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:58:31.265750    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd_odh-ai-gateway-infra(3b573986-98f8-46cf-8b19-cd2f6ec417d8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8"
Jul 17 17:58:31.267095 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:31.267081    2572 scope.go:117] "RemoveContainer" containerID="0d4ea1911424c46d144bab3eed2f3294840f1bf688818e884ccce391f69b590b"
Jul 17 17:58:31.267275 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:58:31.267258    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q_odh-ai-gateway-infra(fa6a5841-7be6-43ff-a37d-f3341997277a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a"
Jul 17 17:58:32.273804 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:32.273776    2572 scope.go:117] "RemoveContainer" containerID="7112f53f17ff7258c5c611cacb0497c7f8dd50af09b0266e80a532acdd1eac28"
Jul 17 17:58:32.274207 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:58:32.273970    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd_odh-ai-gateway-infra(3b573986-98f8-46cf-8b19-cd2f6ec417d8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8"
Jul 17 17:58:43.510304 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:43.510271    2572 scope.go:117] "RemoveContainer" containerID="0d4ea1911424c46d144bab3eed2f3294840f1bf688818e884ccce391f69b590b"
Jul 17 17:58:44.339818 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:44.339783    2572 generic.go:356] "Generic (PLEG): container finished" podID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerID="3349d1d58e3d36adb15a18f0348bf91db418bbe50b9c4bc871b4787779cafce3" exitCode=6
Jul 17 17:58:44.339997 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:44.339852    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q" event={"ID":"fa6a5841-7be6-43ff-a37d-f3341997277a","Type":"ContainerDied","Data":"3349d1d58e3d36adb15a18f0348bf91db418bbe50b9c4bc871b4787779cafce3"}
Jul 17 17:58:44.339997 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:44.339927    2572 scope.go:117] "RemoveContainer" containerID="0d4ea1911424c46d144bab3eed2f3294840f1bf688818e884ccce391f69b590b"
Jul 17 17:58:44.340214 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:44.340185    2572 scope.go:117] "RemoveContainer" containerID="3349d1d58e3d36adb15a18f0348bf91db418bbe50b9c4bc871b4787779cafce3"
Jul 17 17:58:44.340443 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:58:44.340423    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q_odh-ai-gateway-infra(fa6a5841-7be6-43ff-a37d-f3341997277a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a"
Jul 17 17:58:45.376312 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:45.376276    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"]
Jul 17 17:58:45.519070 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:45.519039    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"
Jul 17 17:58:45.601853 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:45.601817    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6a5841-7be6-43ff-a37d-f3341997277a-maas-api-service-ca\") pod \"fa6a5841-7be6-43ff-a37d-f3341997277a\" (UID: \"fa6a5841-7be6-43ff-a37d-f3341997277a\") "
Jul 17 17:58:45.602165 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:45.602143    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa6a5841-7be6-43ff-a37d-f3341997277a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa6a5841-7be6-43ff-a37d-f3341997277a" (UID: "fa6a5841-7be6-43ff-a37d-f3341997277a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:58:45.702554 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:45.702423    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6a5841-7be6-43ff-a37d-f3341997277a-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:58:46.020709 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.020617    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"]
Jul 17 17:58:46.020977 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.020965    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerName="revoke-keys"
Jul 17 17:58:46.021024 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.020979    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerName="revoke-keys"
Jul 17 17:58:46.021024 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.020998    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerName="revoke-keys"
Jul 17 17:58:46.021024 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.021005    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerName="revoke-keys"
Jul 17 17:58:46.021024 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.021012    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerName="revoke-keys"
Jul 17 17:58:46.021024 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.021018    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerName="revoke-keys"
Jul 17 17:58:46.021183 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.021080    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerName="revoke-keys"
Jul 17 17:58:46.021183 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.021087    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerName="revoke-keys"
Jul 17 17:58:46.021183 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.021097    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" containerName="revoke-keys"
Jul 17 17:58:46.024147 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.024126    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"
Jul 17 17:58:46.032532 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.032505    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"]
Jul 17 17:58:46.105304 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.105265    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-86jzm\" (UID: \"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"
Jul 17 17:58:46.205971 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.205924    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-86jzm\" (UID: \"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"
Jul 17 17:58:46.206631 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.206610    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-86jzm\" (UID: \"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"
Jul 17 17:58:46.336632 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.336592    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"
Jul 17 17:58:46.355020 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.354994    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"
Jul 17 17:58:46.355206 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.355037    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q" event={"ID":"fa6a5841-7be6-43ff-a37d-f3341997277a","Type":"ContainerDied","Data":"5faaa2923a0f742c93c2d01e1422a8f4cd5b20981ff7fe864a5b1ab7d52a017c"}
Jul 17 17:58:46.355206 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.355110    2572 scope.go:117] "RemoveContainer" containerID="3349d1d58e3d36adb15a18f0348bf91db418bbe50b9c4bc871b4787779cafce3"
Jul 17 17:58:46.382458 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.382410    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"]
Jul 17 17:58:46.398263 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.398215    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-5cr9q"]
Jul 17 17:58:46.482793 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.482761    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"]
Jul 17 17:58:46.484449 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:58:46.484399    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aa1aa69_dc2a_4b21_a709_eb05d40d6ca8.slice/crio-5a7704889e820cf5c3ad5d123de68023de81aa2eff21069daecc1214990583d4 WatchSource:0}: Error finding container 5a7704889e820cf5c3ad5d123de68023de81aa2eff21069daecc1214990583d4: Status 404 returned error can't find the container with id 5a7704889e820cf5c3ad5d123de68023de81aa2eff21069daecc1214990583d4
Jul 17 17:58:46.516145 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:46.516111    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa6a5841-7be6-43ff-a37d-f3341997277a" path="/var/lib/kubelet/pods/fa6a5841-7be6-43ff-a37d-f3341997277a/volumes"
Jul 17 17:58:47.362569 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:47.362533    2572 generic.go:356] "Generic (PLEG): container finished" podID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerID="d9b4e7ebd88f3df3ffc0ab08c6119fe4a1b30e4e60d59592b3c3f3f61d0458dc" exitCode=7
Jul 17 17:58:47.362771 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:47.362623    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm" event={"ID":"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8","Type":"ContainerDied","Data":"d9b4e7ebd88f3df3ffc0ab08c6119fe4a1b30e4e60d59592b3c3f3f61d0458dc"}
Jul 17 17:58:47.362771 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:47.362705    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm" event={"ID":"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8","Type":"ContainerStarted","Data":"5a7704889e820cf5c3ad5d123de68023de81aa2eff21069daecc1214990583d4"}
Jul 17 17:58:47.362856 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:47.362816    2572 scope.go:117] "RemoveContainer" containerID="d9b4e7ebd88f3df3ffc0ab08c6119fe4a1b30e4e60d59592b3c3f3f61d0458dc"
Jul 17 17:58:47.510591 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:47.510559    2572 scope.go:117] "RemoveContainer" containerID="7112f53f17ff7258c5c611cacb0497c7f8dd50af09b0266e80a532acdd1eac28"
Jul 17 17:58:48.369645 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.369609    2572 generic.go:356] "Generic (PLEG): container finished" podID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerID="75ec0f59286ba9083e7af02695255bbc2c5e5240f50b5bf84024872ca0cbb369" exitCode=6
Jul 17 17:58:48.369834 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.369676    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd" event={"ID":"3b573986-98f8-46cf-8b19-cd2f6ec417d8","Type":"ContainerDied","Data":"75ec0f59286ba9083e7af02695255bbc2c5e5240f50b5bf84024872ca0cbb369"}
Jul 17 17:58:48.369834 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.369739    2572 scope.go:117] "RemoveContainer" containerID="7112f53f17ff7258c5c611cacb0497c7f8dd50af09b0266e80a532acdd1eac28"
Jul 17 17:58:48.370125 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.370108    2572 scope.go:117] "RemoveContainer" containerID="75ec0f59286ba9083e7af02695255bbc2c5e5240f50b5bf84024872ca0cbb369"
Jul 17 17:58:48.370331 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:58:48.370311    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd_odh-ai-gateway-infra(3b573986-98f8-46cf-8b19-cd2f6ec417d8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8"
Jul 17 17:58:48.371980 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.371959    2572 generic.go:356] "Generic (PLEG): container finished" podID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerID="74767920d432c5f530327d31dca1356ee59efb1a4fe8f9012644595f0fde143b" exitCode=7
Jul 17 17:58:48.372080 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.372005    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm" event={"ID":"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8","Type":"ContainerDied","Data":"74767920d432c5f530327d31dca1356ee59efb1a4fe8f9012644595f0fde143b"}
Jul 17 17:58:48.372235 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.372220    2572 scope.go:117] "RemoveContainer" containerID="74767920d432c5f530327d31dca1356ee59efb1a4fe8f9012644595f0fde143b"
Jul 17 17:58:48.372414 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:58:48.372397    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-86jzm_odh-ai-gateway-infra(3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8"
Jul 17 17:58:48.381500 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.381466    2572 scope.go:117] "RemoveContainer" containerID="d9b4e7ebd88f3df3ffc0ab08c6119fe4a1b30e4e60d59592b3c3f3f61d0458dc"
Jul 17 17:58:48.626132 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.626035    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:58:48.630772 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.630745    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 17:58:48.632518 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.632477    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:58:48.636721 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:48.636701    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 17:58:49.379452 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:49.379423    2572 scope.go:117] "RemoveContainer" containerID="74767920d432c5f530327d31dca1356ee59efb1a4fe8f9012644595f0fde143b"
Jul 17 17:58:49.379729 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:58:49.379704    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-86jzm_odh-ai-gateway-infra(3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8"
Jul 17 17:58:49.403942 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:49.403903    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"]
Jul 17 17:58:49.551735 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:49.551702    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"
Jul 17 17:58:49.631396 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:49.631295    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b573986-98f8-46cf-8b19-cd2f6ec417d8-maas-api-service-ca\") pod \"3b573986-98f8-46cf-8b19-cd2f6ec417d8\" (UID: \"3b573986-98f8-46cf-8b19-cd2f6ec417d8\") "
Jul 17 17:58:49.631806 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:49.631658    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b573986-98f8-46cf-8b19-cd2f6ec417d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b573986-98f8-46cf-8b19-cd2f6ec417d8" (UID: "3b573986-98f8-46cf-8b19-cd2f6ec417d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:58:49.731908 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:49.731867    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b573986-98f8-46cf-8b19-cd2f6ec417d8-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:58:50.387378 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:50.387334    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd" event={"ID":"3b573986-98f8-46cf-8b19-cd2f6ec417d8","Type":"ContainerDied","Data":"18895e9886fce947662c9cc8d9bd564ade041d8b0908de7441e2fa44dfa02bae"}
Jul 17 17:58:50.387378 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:50.387358    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"
Jul 17 17:58:50.387656 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:50.387429    2572 scope.go:117] "RemoveContainer" containerID="75ec0f59286ba9083e7af02695255bbc2c5e5240f50b5bf84024872ca0cbb369"
Jul 17 17:58:50.410048 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:50.410011    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"]
Jul 17 17:58:50.416281 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:50.416246    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-4xhpd"]
Jul 17 17:58:50.516355 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:58:50.516325    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" path="/var/lib/kubelet/pods/3b573986-98f8-46cf-8b19-cd2f6ec417d8/volumes"
Jul 17 17:59:03.510270 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:03.510231    2572 scope.go:117] "RemoveContainer" containerID="74767920d432c5f530327d31dca1356ee59efb1a4fe8f9012644595f0fde143b"
Jul 17 17:59:04.471258 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:04.471218    2572 generic.go:356] "Generic (PLEG): container finished" podID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerID="54e05a107b014b1fa3dbed83532b538b1e51fc182b4b7ca65f03466701638404" exitCode=7
Jul 17 17:59:04.471465 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:04.471298    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm" event={"ID":"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8","Type":"ContainerDied","Data":"54e05a107b014b1fa3dbed83532b538b1e51fc182b4b7ca65f03466701638404"}
Jul 17 17:59:04.471465 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:04.471396    2572 scope.go:117] "RemoveContainer" containerID="74767920d432c5f530327d31dca1356ee59efb1a4fe8f9012644595f0fde143b"
Jul 17 17:59:04.471729 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:04.471712    2572 scope.go:117] "RemoveContainer" containerID="54e05a107b014b1fa3dbed83532b538b1e51fc182b4b7ca65f03466701638404"
Jul 17 17:59:04.471939 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:59:04.471918    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-86jzm_odh-ai-gateway-infra(3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8"
Jul 17 17:59:05.499722 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:05.499681    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"]
Jul 17 17:59:05.639574 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:05.639547    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"
Jul 17 17:59:05.670970 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:05.670931    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8-maas-api-service-ca\") pod \"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8\" (UID: \"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8\") "
Jul 17 17:59:05.671244 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:05.671219    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" (UID: "3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:59:05.772701 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:05.772607    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:59:06.485334 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:06.485302    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"
Jul 17 17:59:06.485639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:06.485349    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm" event={"ID":"3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8","Type":"ContainerDied","Data":"5a7704889e820cf5c3ad5d123de68023de81aa2eff21069daecc1214990583d4"}
Jul 17 17:59:06.485639 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:06.485409    2572 scope.go:117] "RemoveContainer" containerID="54e05a107b014b1fa3dbed83532b538b1e51fc182b4b7ca65f03466701638404"
Jul 17 17:59:06.507961 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:06.507922    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"]
Jul 17 17:59:06.516670 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:06.516641    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-86jzm"]
Jul 17 17:59:08.516404 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:08.516367    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" path="/var/lib/kubelet/pods/3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8/volumes"
Jul 17 17:59:17.577338 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577293    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"]
Jul 17 17:59:17.577885 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577851    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerName="revoke-keys"
Jul 17 17:59:17.577885 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577870    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerName="revoke-keys"
Jul 17 17:59:17.577885 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577885    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerName="revoke-keys"
Jul 17 17:59:17.578063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577893    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerName="revoke-keys"
Jul 17 17:59:17.578063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577920    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerName="revoke-keys"
Jul 17 17:59:17.578063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577930    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerName="revoke-keys"
Jul 17 17:59:17.578063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577942    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerName="revoke-keys"
Jul 17 17:59:17.578063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577950    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerName="revoke-keys"
Jul 17 17:59:17.578063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577967    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerName="revoke-keys"
Jul 17 17:59:17.578063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.577975    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerName="revoke-keys"
Jul 17 17:59:17.578063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.578054    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerName="revoke-keys"
Jul 17 17:59:17.578445 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.578065    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerName="revoke-keys"
Jul 17 17:59:17.578445 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.578075    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerName="revoke-keys"
Jul 17 17:59:17.578445 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.578084    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerName="revoke-keys"
Jul 17 17:59:17.578445 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.578094    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa1aa69-dc2a-4b21-a709-eb05d40d6ca8" containerName="revoke-keys"
Jul 17 17:59:17.582716 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.582688    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"
Jul 17 17:59:17.585600 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.585572    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 17:59:17.585757 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.585585    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 17:59:17.591646 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.591618    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"]
Jul 17 17:59:17.678287 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.678239    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fb1329e-3887-4a02-bbf9-678ee82e1e9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6\" (UID: \"1fb1329e-3887-4a02-bbf9-678ee82e1e9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"
Jul 17 17:59:17.779080 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.779036    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fb1329e-3887-4a02-bbf9-678ee82e1e9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6\" (UID: \"1fb1329e-3887-4a02-bbf9-678ee82e1e9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"
Jul 17 17:59:17.779720 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.779701    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fb1329e-3887-4a02-bbf9-678ee82e1e9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6\" (UID: \"1fb1329e-3887-4a02-bbf9-678ee82e1e9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"
Jul 17 17:59:17.895397 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:17.895297    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"
Jul 17 17:59:18.036032 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:18.036000    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"]
Jul 17 17:59:18.038782 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:59:18.038750    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fb1329e_3887_4a02_bbf9_678ee82e1e9f.slice/crio-7d7f0f2147de0397bd900b7918509ed4fdb902c1b3ce0875c52549297141e372 WatchSource:0}: Error finding container 7d7f0f2147de0397bd900b7918509ed4fdb902c1b3ce0875c52549297141e372: Status 404 returned error can't find the container with id 7d7f0f2147de0397bd900b7918509ed4fdb902c1b3ce0875c52549297141e372
Jul 17 17:59:18.556446 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:18.556404    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" event={"ID":"1fb1329e-3887-4a02-bbf9-678ee82e1e9f","Type":"ContainerStarted","Data":"4a8f20e67c3e3ff385c24cd5f6fc0ad5a990a6d4fb21201af569fa7ed151e3f6"}
Jul 17 17:59:18.556653 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:18.556469    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" event={"ID":"1fb1329e-3887-4a02-bbf9-678ee82e1e9f","Type":"ContainerStarted","Data":"7d7f0f2147de0397bd900b7918509ed4fdb902c1b3ce0875c52549297141e372"}
Jul 17 17:59:18.581367 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:18.581307    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" podStartSLOduration=1.5812869 podStartE2EDuration="1.5812869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:59:18.578715448 +0000 UTC m=+2130.584071297" watchObservedRunningTime="2026-07-17 17:59:18.5812869 +0000 UTC m=+2130.586642817"
Jul 17 17:59:19.563429 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:19.563391    2572 generic.go:356] "Generic (PLEG): container finished" podID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerID="4a8f20e67c3e3ff385c24cd5f6fc0ad5a990a6d4fb21201af569fa7ed151e3f6" exitCode=6
Jul 17 17:59:19.563691 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:19.563483    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" event={"ID":"1fb1329e-3887-4a02-bbf9-678ee82e1e9f","Type":"ContainerDied","Data":"4a8f20e67c3e3ff385c24cd5f6fc0ad5a990a6d4fb21201af569fa7ed151e3f6"}
Jul 17 17:59:19.563777 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:19.563758    2572 scope.go:117] "RemoveContainer" containerID="4a8f20e67c3e3ff385c24cd5f6fc0ad5a990a6d4fb21201af569fa7ed151e3f6"
Jul 17 17:59:20.571359 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:20.571318    2572 generic.go:356] "Generic (PLEG): container finished" podID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerID="432efc50f6d476998db2107540f49275d23fc34b804fdb4fb7c533892976da58" exitCode=6
Jul 17 17:59:20.571873 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:20.571416    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" event={"ID":"1fb1329e-3887-4a02-bbf9-678ee82e1e9f","Type":"ContainerDied","Data":"432efc50f6d476998db2107540f49275d23fc34b804fdb4fb7c533892976da58"}
Jul 17 17:59:20.571873 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:20.571505    2572 scope.go:117] "RemoveContainer" containerID="4a8f20e67c3e3ff385c24cd5f6fc0ad5a990a6d4fb21201af569fa7ed151e3f6"
Jul 17 17:59:20.571873 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:20.571632    2572 scope.go:117] "RemoveContainer" containerID="432efc50f6d476998db2107540f49275d23fc34b804fdb4fb7c533892976da58"
Jul 17 17:59:20.571873 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:59:20.571845    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6_odh-ai-gateway-infra(1fb1329e-3887-4a02-bbf9-678ee82e1e9f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f"
Jul 17 17:59:21.578791 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:21.578760    2572 scope.go:117] "RemoveContainer" containerID="432efc50f6d476998db2107540f49275d23fc34b804fdb4fb7c533892976da58"
Jul 17 17:59:21.579194 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:59:21.578962    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6_odh-ai-gateway-infra(1fb1329e-3887-4a02-bbf9-678ee82e1e9f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f"
Jul 17 17:59:29.122456 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.122411    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"]
Jul 17 17:59:29.122879 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.122825    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerName="revoke-keys"
Jul 17 17:59:29.122879 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.122838    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerName="revoke-keys"
Jul 17 17:59:29.122970 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.122926    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b573986-98f8-46cf-8b19-cd2f6ec417d8" containerName="revoke-keys"
Jul 17 17:59:29.125866 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.125846    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"
Jul 17 17:59:29.145478 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.145437    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"]
Jul 17 17:59:29.177618 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.177573    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3177bcc0-e2cf-4748-bec9-88f2360f4f08-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g\" (UID: \"3177bcc0-e2cf-4748-bec9-88f2360f4f08\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"
Jul 17 17:59:29.278126 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.278084    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3177bcc0-e2cf-4748-bec9-88f2360f4f08-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g\" (UID: \"3177bcc0-e2cf-4748-bec9-88f2360f4f08\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"
Jul 17 17:59:29.278854 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.278825    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3177bcc0-e2cf-4748-bec9-88f2360f4f08-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g\" (UID: \"3177bcc0-e2cf-4748-bec9-88f2360f4f08\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"
Jul 17 17:59:29.435529 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.435391    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"
Jul 17 17:59:29.579098 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.579063    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"]
Jul 17 17:59:29.581592 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:59:29.581563    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3177bcc0_e2cf_4748_bec9_88f2360f4f08.slice/crio-f423f21a4058eca1ac54cf88dd0ac6dd1c24684724fb13c4c05043264fd59c46 WatchSource:0}: Error finding container f423f21a4058eca1ac54cf88dd0ac6dd1c24684724fb13c4c05043264fd59c46: Status 404 returned error can't find the container with id f423f21a4058eca1ac54cf88dd0ac6dd1c24684724fb13c4c05043264fd59c46
Jul 17 17:59:29.622212 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:29.622171    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g" event={"ID":"3177bcc0-e2cf-4748-bec9-88f2360f4f08","Type":"ContainerStarted","Data":"f423f21a4058eca1ac54cf88dd0ac6dd1c24684724fb13c4c05043264fd59c46"}
Jul 17 17:59:30.629246 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:30.629207    2572 generic.go:356] "Generic (PLEG): container finished" podID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerID="749641e9bd46a9c4944fc38bab48a86d41fc22e7f52a3d89b7467d87251feef5" exitCode=6
Jul 17 17:59:30.629797 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:30.629298    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g" event={"ID":"3177bcc0-e2cf-4748-bec9-88f2360f4f08","Type":"ContainerDied","Data":"749641e9bd46a9c4944fc38bab48a86d41fc22e7f52a3d89b7467d87251feef5"}
Jul 17 17:59:30.629797 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:30.629526    2572 scope.go:117] "RemoveContainer" containerID="749641e9bd46a9c4944fc38bab48a86d41fc22e7f52a3d89b7467d87251feef5"
Jul 17 17:59:31.636535 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:31.636415    2572 generic.go:356] "Generic (PLEG): container finished" podID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerID="7dab4396108a52696fbdc437de55bfcb293f19bef12a329a9eccc75e7612faca" exitCode=6
Jul 17 17:59:31.636535 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:31.636516    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g" event={"ID":"3177bcc0-e2cf-4748-bec9-88f2360f4f08","Type":"ContainerDied","Data":"7dab4396108a52696fbdc437de55bfcb293f19bef12a329a9eccc75e7612faca"}
Jul 17 17:59:31.637090 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:31.636591    2572 scope.go:117] "RemoveContainer" containerID="749641e9bd46a9c4944fc38bab48a86d41fc22e7f52a3d89b7467d87251feef5"
Jul 17 17:59:31.637090 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:31.636862    2572 scope.go:117] "RemoveContainer" containerID="7dab4396108a52696fbdc437de55bfcb293f19bef12a329a9eccc75e7612faca"
Jul 17 17:59:31.637090 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:59:31.637080    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g_odh-ai-gateway-infra(3177bcc0-e2cf-4748-bec9-88f2360f4f08)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08"
Jul 17 17:59:32.509922 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:32.509886    2572 scope.go:117] "RemoveContainer" containerID="432efc50f6d476998db2107540f49275d23fc34b804fdb4fb7c533892976da58"
Jul 17 17:59:32.643753 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:32.643725    2572 scope.go:117] "RemoveContainer" containerID="7dab4396108a52696fbdc437de55bfcb293f19bef12a329a9eccc75e7612faca"
Jul 17 17:59:32.644167 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:59:32.643960    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g_odh-ai-gateway-infra(3177bcc0-e2cf-4748-bec9-88f2360f4f08)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08"
Jul 17 17:59:33.650420 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:33.650385    2572 generic.go:356] "Generic (PLEG): container finished" podID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerID="2639bc9c5b01f9288837edeb138d50b1dd7cbee5ccf063040b20d2ea7c5ffba9" exitCode=6
Jul 17 17:59:33.650906 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:33.650457    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" event={"ID":"1fb1329e-3887-4a02-bbf9-678ee82e1e9f","Type":"ContainerDied","Data":"2639bc9c5b01f9288837edeb138d50b1dd7cbee5ccf063040b20d2ea7c5ffba9"}
Jul 17 17:59:33.650906 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:33.650543    2572 scope.go:117] "RemoveContainer" containerID="432efc50f6d476998db2107540f49275d23fc34b804fdb4fb7c533892976da58"
Jul 17 17:59:33.650906 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:33.650825    2572 scope.go:117] "RemoveContainer" containerID="2639bc9c5b01f9288837edeb138d50b1dd7cbee5ccf063040b20d2ea7c5ffba9"
Jul 17 17:59:33.651084 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:59:33.651062    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6_odh-ai-gateway-infra(1fb1329e-3887-4a02-bbf9-678ee82e1e9f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f"
Jul 17 17:59:34.678661 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:34.678622    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"]
Jul 17 17:59:34.825047 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:34.825020    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"
Jul 17 17:59:34.929220 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:34.929126    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fb1329e-3887-4a02-bbf9-678ee82e1e9f-maas-api-service-ca\") pod \"1fb1329e-3887-4a02-bbf9-678ee82e1e9f\" (UID: \"1fb1329e-3887-4a02-bbf9-678ee82e1e9f\") "
Jul 17 17:59:34.929554 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:34.929527    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fb1329e-3887-4a02-bbf9-678ee82e1e9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1fb1329e-3887-4a02-bbf9-678ee82e1e9f" (UID: "1fb1329e-3887-4a02-bbf9-678ee82e1e9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:59:35.030708 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:35.030667    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fb1329e-3887-4a02-bbf9-678ee82e1e9f-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:59:35.664445 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:35.664417    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"
Jul 17 17:59:35.664661 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:35.664412    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6" event={"ID":"1fb1329e-3887-4a02-bbf9-678ee82e1e9f","Type":"ContainerDied","Data":"7d7f0f2147de0397bd900b7918509ed4fdb902c1b3ce0875c52549297141e372"}
Jul 17 17:59:35.664661 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:35.664560    2572 scope.go:117] "RemoveContainer" containerID="2639bc9c5b01f9288837edeb138d50b1dd7cbee5ccf063040b20d2ea7c5ffba9"
Jul 17 17:59:35.708344 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:35.708308    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"]
Jul 17 17:59:35.712932 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:35.712902    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-tmfn6"]
Jul 17 17:59:36.515349 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:36.515314    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" path="/var/lib/kubelet/pods/1fb1329e-3887-4a02-bbf9-678ee82e1e9f/volumes"
Jul 17 17:59:44.510081 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:44.509985    2572 scope.go:117] "RemoveContainer" containerID="7dab4396108a52696fbdc437de55bfcb293f19bef12a329a9eccc75e7612faca"
Jul 17 17:59:45.726830 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:45.726786    2572 generic.go:356] "Generic (PLEG): container finished" podID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerID="caa552aef92b007f681937633684ddc3c60447a9eb06e3f0dea636b5cb7cf8af" exitCode=6
Jul 17 17:59:45.727291 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:45.726858    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g" event={"ID":"3177bcc0-e2cf-4748-bec9-88f2360f4f08","Type":"ContainerDied","Data":"caa552aef92b007f681937633684ddc3c60447a9eb06e3f0dea636b5cb7cf8af"}
Jul 17 17:59:45.727291 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:45.726947    2572 scope.go:117] "RemoveContainer" containerID="7dab4396108a52696fbdc437de55bfcb293f19bef12a329a9eccc75e7612faca"
Jul 17 17:59:45.727291 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:45.727242    2572 scope.go:117] "RemoveContainer" containerID="caa552aef92b007f681937633684ddc3c60447a9eb06e3f0dea636b5cb7cf8af"
Jul 17 17:59:45.727480 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:59:45.727456    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g_odh-ai-gateway-infra(3177bcc0-e2cf-4748-bec9-88f2360f4f08)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08"
Jul 17 17:59:46.064808 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.064769    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"]
Jul 17 17:59:46.065159 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.065144    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerName="revoke-keys"
Jul 17 17:59:46.065204 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.065162    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerName="revoke-keys"
Jul 17 17:59:46.065204 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.065180    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerName="revoke-keys"
Jul 17 17:59:46.065204 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.065185    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerName="revoke-keys"
Jul 17 17:59:46.065300 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.065253    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerName="revoke-keys"
Jul 17 17:59:46.065300 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.065263    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerName="revoke-keys"
Jul 17 17:59:46.065300 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.065269    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerName="revoke-keys"
Jul 17 17:59:46.069622 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.069603    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"
Jul 17 17:59:46.075841 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.075811    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"]
Jul 17 17:59:46.230542 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.230447    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26efc90-cfa9-48ba-9542-f80d8fb1cdb5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-rt964\" (UID: \"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"
Jul 17 17:59:46.331561 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.331451    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26efc90-cfa9-48ba-9542-f80d8fb1cdb5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-rt964\" (UID: \"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"
Jul 17 17:59:46.332126 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.332106    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26efc90-cfa9-48ba-9542-f80d8fb1cdb5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-rt964\" (UID: \"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"
Jul 17 17:59:46.383015 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.382979    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"
Jul 17 17:59:46.531034 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.531011    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"]
Jul 17 17:59:46.532037 ip-10-0-135-30 kubenswrapper[2572]: W0717 17:59:46.532007    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26efc90_cfa9_48ba_9542_f80d8fb1cdb5.slice/crio-06ce6f7da87534cbb6050df5ba1c389d34cce4c861ba0feeaaa35aa46c77deb4 WatchSource:0}: Error finding container 06ce6f7da87534cbb6050df5ba1c389d34cce4c861ba0feeaaa35aa46c77deb4: Status 404 returned error can't find the container with id 06ce6f7da87534cbb6050df5ba1c389d34cce4c861ba0feeaaa35aa46c77deb4
Jul 17 17:59:46.735040 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.734999    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" event={"ID":"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5","Type":"ContainerStarted","Data":"06ce6f7da87534cbb6050df5ba1c389d34cce4c861ba0feeaaa35aa46c77deb4"}
Jul 17 17:59:46.757019 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.756984    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"]
Jul 17 17:59:46.893790 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:46.893717    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"
Jul 17 17:59:47.036413 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.036360    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3177bcc0-e2cf-4748-bec9-88f2360f4f08-maas-api-service-ca\") pod \"3177bcc0-e2cf-4748-bec9-88f2360f4f08\" (UID: \"3177bcc0-e2cf-4748-bec9-88f2360f4f08\") "
Jul 17 17:59:47.036807 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.036784    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3177bcc0-e2cf-4748-bec9-88f2360f4f08-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3177bcc0-e2cf-4748-bec9-88f2360f4f08" (UID: "3177bcc0-e2cf-4748-bec9-88f2360f4f08"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:59:47.137463 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.137425    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3177bcc0-e2cf-4748-bec9-88f2360f4f08-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 17:59:47.741620 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.741593    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"
Jul 17 17:59:47.741620 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.741598    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g" event={"ID":"3177bcc0-e2cf-4748-bec9-88f2360f4f08","Type":"ContainerDied","Data":"f423f21a4058eca1ac54cf88dd0ac6dd1c24684724fb13c4c05043264fd59c46"}
Jul 17 17:59:47.742150 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.741668    2572 scope.go:117] "RemoveContainer" containerID="caa552aef92b007f681937633684ddc3c60447a9eb06e3f0dea636b5cb7cf8af"
Jul 17 17:59:47.743301 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.743270    2572 generic.go:356] "Generic (PLEG): container finished" podID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerID="8488b63d4f3351ed9a834dd5ea9fe23bd116f3520719bb9a055d23ce79d24d75" exitCode=7
Jul 17 17:59:47.743434 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.743394    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" event={"ID":"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5","Type":"ContainerDied","Data":"8488b63d4f3351ed9a834dd5ea9fe23bd116f3520719bb9a055d23ce79d24d75"}
Jul 17 17:59:47.743578 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.743554    2572 scope.go:117] "RemoveContainer" containerID="8488b63d4f3351ed9a834dd5ea9fe23bd116f3520719bb9a055d23ce79d24d75"
Jul 17 17:59:47.780923 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.780887    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"]
Jul 17 17:59:47.786305 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:47.786276    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-p8j2g"]
Jul 17 17:59:48.515289 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:48.515258    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" path="/var/lib/kubelet/pods/3177bcc0-e2cf-4748-bec9-88f2360f4f08/volumes"
Jul 17 17:59:48.751605 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:48.751513    2572 generic.go:356] "Generic (PLEG): container finished" podID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerID="b6fcdd4fd7eb36ecda2a7530b574b140d40575971a6ba1fbb7c6294643ac5a95" exitCode=7
Jul 17 17:59:48.751605 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:48.751593    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" event={"ID":"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5","Type":"ContainerDied","Data":"b6fcdd4fd7eb36ecda2a7530b574b140d40575971a6ba1fbb7c6294643ac5a95"}
Jul 17 17:59:48.752063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:48.751653    2572 scope.go:117] "RemoveContainer" containerID="8488b63d4f3351ed9a834dd5ea9fe23bd116f3520719bb9a055d23ce79d24d75"
Jul 17 17:59:48.752063 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:48.751860    2572 scope.go:117] "RemoveContainer" containerID="b6fcdd4fd7eb36ecda2a7530b574b140d40575971a6ba1fbb7c6294643ac5a95"
Jul 17 17:59:48.752142 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:59:48.752095    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-rt964_odh-ai-gateway-infra(e26efc90-cfa9-48ba-9542-f80d8fb1cdb5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5"
Jul 17 17:59:49.758018 ip-10-0-135-30 kubenswrapper[2572]: I0717 17:59:49.757985    2572 scope.go:117] "RemoveContainer" containerID="b6fcdd4fd7eb36ecda2a7530b574b140d40575971a6ba1fbb7c6294643ac5a95"
Jul 17 17:59:49.758478 ip-10-0-135-30 kubenswrapper[2572]: E0717 17:59:49.758171    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-rt964_odh-ai-gateway-infra(e26efc90-cfa9-48ba-9542-f80d8fb1cdb5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5"
Jul 17 18:00:00.138887 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.138832    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f"]
Jul 17 18:00:00.139425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139303    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerName="revoke-keys"
Jul 17 18:00:00.139425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139319    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerName="revoke-keys"
Jul 17 18:00:00.139425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139329    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerName="revoke-keys"
Jul 17 18:00:00.139425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139336    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fb1329e-3887-4a02-bbf9-678ee82e1e9f" containerName="revoke-keys"
Jul 17 18:00:00.139425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139348    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerName="revoke-keys"
Jul 17 18:00:00.139425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139359    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerName="revoke-keys"
Jul 17 18:00:00.139425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139368    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerName="revoke-keys"
Jul 17 18:00:00.139425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139376    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerName="revoke-keys"
Jul 17 18:00:00.139866 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139467    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerName="revoke-keys"
Jul 17 18:00:00.139866 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.139484    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerName="revoke-keys"
Jul 17 18:00:00.142423 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.142399    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"]
Jul 17 18:00:00.142570 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.142551    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f"
Jul 17 18:00:00.142789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.142776    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3177bcc0-e2cf-4748-bec9-88f2360f4f08" containerName="revoke-keys"
Jul 17 18:00:00.145637 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.145621    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"
Jul 17 18:00:00.150770 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.150535    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f"]
Jul 17 18:00:00.154064 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.154027    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"]
Jul 17 18:00:00.156012 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.155992    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f"
Jul 17 18:00:00.164373 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.164352    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"
Jul 17 18:00:00.356203 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.356052    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"]
Jul 17 18:00:00.358572 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:00:00.358542    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89bb8811_92c7_433d_a015_985f290957db.slice/crio-ae654c2d0445ebecad32356ed3b50a24af5d63c1743c3a2d1ae20764f1a9e11e WatchSource:0}: Error finding container ae654c2d0445ebecad32356ed3b50a24af5d63c1743c3a2d1ae20764f1a9e11e: Status 404 returned error can't find the container with id ae654c2d0445ebecad32356ed3b50a24af5d63c1743c3a2d1ae20764f1a9e11e
Jul 17 18:00:00.542064 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.542040    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f"]
Jul 17 18:00:00.543999 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:00:00.543971    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd266de_a508_49f5_bb09_eb3508990517.slice/crio-fc0cf9fdaafd183a175af5ba1e43688d8703382a9a81320e073b4ff860ad1fb0 WatchSource:0}: Error finding container fc0cf9fdaafd183a175af5ba1e43688d8703382a9a81320e073b4ff860ad1fb0: Status 404 returned error can't find the container with id fc0cf9fdaafd183a175af5ba1e43688d8703382a9a81320e073b4ff860ad1fb0
Jul 17 18:00:00.814395 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.814358    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f" event={"ID":"7bd266de-a508-49f5-bb09-eb3508990517","Type":"ContainerStarted","Data":"fc0cf9fdaafd183a175af5ba1e43688d8703382a9a81320e073b4ff860ad1fb0"}
Jul 17 18:00:00.816054 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.816020    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" event={"ID":"89bb8811-92c7-433d-a015-985f290957db","Type":"ContainerStarted","Data":"d10fa1321fa075a4fee9dcfe3c296d52e441ce5f14c6371d2fa3a5e937d05bc3"}
Jul 17 18:00:00.816223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.816076    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" event={"ID":"89bb8811-92c7-433d-a015-985f290957db","Type":"ContainerStarted","Data":"ae654c2d0445ebecad32356ed3b50a24af5d63c1743c3a2d1ae20764f1a9e11e"}
Jul 17 18:00:00.837348 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:00.837289    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" podStartSLOduration=0.837270397 podStartE2EDuration="837.270397ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:00:00.835037071 +0000 UTC m=+2172.840392946" watchObservedRunningTime="2026-07-17 18:00:00.837270397 +0000 UTC m=+2172.842626249"
Jul 17 18:00:01.823124 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:01.823087    2572 generic.go:356] "Generic (PLEG): container finished" podID="7bd266de-a508-49f5-bb09-eb3508990517" containerID="66ab0967c1024940649fb73b1e0f56718c7f5fd0156b5badd2ce8e9ff85d6d2f" exitCode=0
Jul 17 18:00:01.823635 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:01.823168    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f" event={"ID":"7bd266de-a508-49f5-bb09-eb3508990517","Type":"ContainerDied","Data":"66ab0967c1024940649fb73b1e0f56718c7f5fd0156b5badd2ce8e9ff85d6d2f"}
Jul 17 18:00:01.824792 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:01.824766    2572 generic.go:356] "Generic (PLEG): container finished" podID="89bb8811-92c7-433d-a015-985f290957db" containerID="d10fa1321fa075a4fee9dcfe3c296d52e441ce5f14c6371d2fa3a5e937d05bc3" exitCode=7
Jul 17 18:00:01.824897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:01.824806    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" event={"ID":"89bb8811-92c7-433d-a015-985f290957db","Type":"ContainerDied","Data":"d10fa1321fa075a4fee9dcfe3c296d52e441ce5f14c6371d2fa3a5e937d05bc3"}
Jul 17 18:00:01.824985 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:01.824974    2572 scope.go:117] "RemoveContainer" containerID="d10fa1321fa075a4fee9dcfe3c296d52e441ce5f14c6371d2fa3a5e937d05bc3"
Jul 17 18:00:02.832320 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:02.832285    2572 generic.go:356] "Generic (PLEG): container finished" podID="89bb8811-92c7-433d-a015-985f290957db" containerID="8a7b96e6ab6261960b1b0c917b4d5f1bb296e9322d5be8c6413725ed0e02019e" exitCode=7
Jul 17 18:00:02.832821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:02.832376    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" event={"ID":"89bb8811-92c7-433d-a015-985f290957db","Type":"ContainerDied","Data":"8a7b96e6ab6261960b1b0c917b4d5f1bb296e9322d5be8c6413725ed0e02019e"}
Jul 17 18:00:02.832821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:02.832450    2572 scope.go:117] "RemoveContainer" containerID="d10fa1321fa075a4fee9dcfe3c296d52e441ce5f14c6371d2fa3a5e937d05bc3"
Jul 17 18:00:02.832821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:02.832556    2572 scope.go:117] "RemoveContainer" containerID="8a7b96e6ab6261960b1b0c917b4d5f1bb296e9322d5be8c6413725ed0e02019e"
Jul 17 18:00:02.832821 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:02.832797    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf_odh-ai-gateway-infra(89bb8811-92c7-433d-a015-985f290957db)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" podUID="89bb8811-92c7-433d-a015-985f290957db"
Jul 17 18:00:02.979904 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:02.979872    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f"
Jul 17 18:00:03.510037 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:03.509999    2572 scope.go:117] "RemoveContainer" containerID="b6fcdd4fd7eb36ecda2a7530b574b140d40575971a6ba1fbb7c6294643ac5a95"
Jul 17 18:00:03.840542 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:03.840484    2572 scope.go:117] "RemoveContainer" containerID="8a7b96e6ab6261960b1b0c917b4d5f1bb296e9322d5be8c6413725ed0e02019e"
Jul 17 18:00:03.840998 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:03.840757    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf_odh-ai-gateway-infra(89bb8811-92c7-433d-a015-985f290957db)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" podUID="89bb8811-92c7-433d-a015-985f290957db"
Jul 17 18:00:03.841710 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:03.841693    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f"
Jul 17 18:00:03.841766 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:03.841728    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738520-v9r5f" event={"ID":"7bd266de-a508-49f5-bb09-eb3508990517","Type":"ContainerDied","Data":"fc0cf9fdaafd183a175af5ba1e43688d8703382a9a81320e073b4ff860ad1fb0"}
Jul 17 18:00:03.841810 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:03.841765    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc0cf9fdaafd183a175af5ba1e43688d8703382a9a81320e073b4ff860ad1fb0"
Jul 17 18:00:03.843823 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:03.843800    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" event={"ID":"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5","Type":"ContainerStarted","Data":"f1cd20c97c846e606ec388e01e579ebb5151156dfa11c4ea55d8196f541e5f1b"}
Jul 17 18:00:03.900636 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:03.900578    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" podStartSLOduration=17.900556331 podStartE2EDuration="17.900556331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:59:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:00:03.897867679 +0000 UTC m=+2175.903223533" watchObservedRunningTime="2026-07-17 18:00:03.900556331 +0000 UTC m=+2175.905912184"
Jul 17 18:00:04.850762 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:04.850727    2572 generic.go:356] "Generic (PLEG): container finished" podID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerID="f1cd20c97c846e606ec388e01e579ebb5151156dfa11c4ea55d8196f541e5f1b" exitCode=7
Jul 17 18:00:04.851167 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:04.850803    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" event={"ID":"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5","Type":"ContainerDied","Data":"f1cd20c97c846e606ec388e01e579ebb5151156dfa11c4ea55d8196f541e5f1b"}
Jul 17 18:00:04.851167 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:04.850867    2572 scope.go:117] "RemoveContainer" containerID="b6fcdd4fd7eb36ecda2a7530b574b140d40575971a6ba1fbb7c6294643ac5a95"
Jul 17 18:00:04.851167 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:04.851151    2572 scope.go:117] "RemoveContainer" containerID="f1cd20c97c846e606ec388e01e579ebb5151156dfa11c4ea55d8196f541e5f1b"
Jul 17 18:00:04.851364 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:04.851342    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-rt964_odh-ai-gateway-infra(e26efc90-cfa9-48ba-9542-f80d8fb1cdb5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5"
Jul 17 18:00:04.908426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:04.908390    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"]
Jul 17 18:00:05.996759 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:05.996729    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"
Jul 17 18:00:06.092808 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:06.092772    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26efc90-cfa9-48ba-9542-f80d8fb1cdb5-maas-api-service-ca\") pod \"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5\" (UID: \"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5\") "
Jul 17 18:00:06.093183 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:06.093156    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e26efc90-cfa9-48ba-9542-f80d8fb1cdb5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" (UID: "e26efc90-cfa9-48ba-9542-f80d8fb1cdb5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:00:06.193567 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:06.193433    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26efc90-cfa9-48ba-9542-f80d8fb1cdb5-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:00:06.865415 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:06.865383    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"
Jul 17 18:00:06.865634 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:06.865415    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964" event={"ID":"e26efc90-cfa9-48ba-9542-f80d8fb1cdb5","Type":"ContainerDied","Data":"06ce6f7da87534cbb6050df5ba1c389d34cce4c861ba0feeaaa35aa46c77deb4"}
Jul 17 18:00:06.865634 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:06.865476    2572 scope.go:117] "RemoveContainer" containerID="f1cd20c97c846e606ec388e01e579ebb5151156dfa11c4ea55d8196f541e5f1b"
Jul 17 18:00:06.883423 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:06.883385    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"]
Jul 17 18:00:06.885954 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:06.885927    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rt964"]
Jul 17 18:00:07.621408 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621371    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"]
Jul 17 18:00:07.621858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621741    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd266de-a508-49f5-bb09-eb3508990517" containerName="cleanup"
Jul 17 18:00:07.621858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621756    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd266de-a508-49f5-bb09-eb3508990517" containerName="cleanup"
Jul 17 18:00:07.621858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621769    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerName="revoke-keys"
Jul 17 18:00:07.621858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621775    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerName="revoke-keys"
Jul 17 18:00:07.621858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621784    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerName="revoke-keys"
Jul 17 18:00:07.621858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621789    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerName="revoke-keys"
Jul 17 18:00:07.621858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621802    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerName="revoke-keys"
Jul 17 18:00:07.621858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621806    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerName="revoke-keys"
Jul 17 18:00:07.621858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621859    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerName="revoke-keys"
Jul 17 18:00:07.622138 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621868    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bd266de-a508-49f5-bb09-eb3508990517" containerName="cleanup"
Jul 17 18:00:07.622138 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.621877    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerName="revoke-keys"
Jul 17 18:00:07.626778 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.626753    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"
Jul 17 18:00:07.630071 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.630039    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:00:07.632800 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.632770    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"]
Jul 17 18:00:07.705749 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.705706    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee4ff7f-32da-43f4-a96f-17a37415629c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld\" (UID: \"0ee4ff7f-32da-43f4-a96f-17a37415629c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"
Jul 17 18:00:07.806762 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.806713    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee4ff7f-32da-43f4-a96f-17a37415629c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld\" (UID: \"0ee4ff7f-32da-43f4-a96f-17a37415629c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"
Jul 17 18:00:07.807420 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.807390    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee4ff7f-32da-43f4-a96f-17a37415629c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld\" (UID: \"0ee4ff7f-32da-43f4-a96f-17a37415629c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"
Jul 17 18:00:07.939408 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:07.939316    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"
Jul 17 18:00:08.084242 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:08.084201    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"]
Jul 17 18:00:08.087594 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:00:08.087561    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee4ff7f_32da_43f4_a96f_17a37415629c.slice/crio-5512e2df2ff5ea1e2681f0220aeb3b393edbba8241b4245fafdd966c426ac60d WatchSource:0}: Error finding container 5512e2df2ff5ea1e2681f0220aeb3b393edbba8241b4245fafdd966c426ac60d: Status 404 returned error can't find the container with id 5512e2df2ff5ea1e2681f0220aeb3b393edbba8241b4245fafdd966c426ac60d
Jul 17 18:00:08.517199 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:08.517118    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" path="/var/lib/kubelet/pods/e26efc90-cfa9-48ba-9542-f80d8fb1cdb5/volumes"
Jul 17 18:00:08.879011 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:08.878977    2572 generic.go:356] "Generic (PLEG): container finished" podID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerID="8ab442536c328ded1e32e836d40de8ce5dfb8e01b1b4e40029f45f96a7a747f3" exitCode=6
Jul 17 18:00:08.879414 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:08.879044    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" event={"ID":"0ee4ff7f-32da-43f4-a96f-17a37415629c","Type":"ContainerDied","Data":"8ab442536c328ded1e32e836d40de8ce5dfb8e01b1b4e40029f45f96a7a747f3"}
Jul 17 18:00:08.879414 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:08.879083    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" event={"ID":"0ee4ff7f-32da-43f4-a96f-17a37415629c","Type":"ContainerStarted","Data":"5512e2df2ff5ea1e2681f0220aeb3b393edbba8241b4245fafdd966c426ac60d"}
Jul 17 18:00:08.879414 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:08.879243    2572 scope.go:117] "RemoveContainer" containerID="8ab442536c328ded1e32e836d40de8ce5dfb8e01b1b4e40029f45f96a7a747f3"
Jul 17 18:00:09.887068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:09.887025    2572 generic.go:356] "Generic (PLEG): container finished" podID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerID="ccd3c144d96c42142fa35f5cfb1c53e5e05485c75b9bbbcb1a917c6e6df11aa9" exitCode=6
Jul 17 18:00:09.887529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:09.887112    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" event={"ID":"0ee4ff7f-32da-43f4-a96f-17a37415629c","Type":"ContainerDied","Data":"ccd3c144d96c42142fa35f5cfb1c53e5e05485c75b9bbbcb1a917c6e6df11aa9"}
Jul 17 18:00:09.887529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:09.887210    2572 scope.go:117] "RemoveContainer" containerID="8ab442536c328ded1e32e836d40de8ce5dfb8e01b1b4e40029f45f96a7a747f3"
Jul 17 18:00:09.887529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:09.887419    2572 scope.go:117] "RemoveContainer" containerID="ccd3c144d96c42142fa35f5cfb1c53e5e05485c75b9bbbcb1a917c6e6df11aa9"
Jul 17 18:00:09.887711 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:09.887692    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld_odh-ai-gateway-infra(0ee4ff7f-32da-43f4-a96f-17a37415629c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c"
Jul 17 18:00:10.894403 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:10.894365    2572 scope.go:117] "RemoveContainer" containerID="ccd3c144d96c42142fa35f5cfb1c53e5e05485c75b9bbbcb1a917c6e6df11aa9"
Jul 17 18:00:10.894865 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:10.894719    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld_odh-ai-gateway-infra(0ee4ff7f-32da-43f4-a96f-17a37415629c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c"
Jul 17 18:00:15.510010 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:15.509974    2572 scope.go:117] "RemoveContainer" containerID="8a7b96e6ab6261960b1b0c917b4d5f1bb296e9322d5be8c6413725ed0e02019e"
Jul 17 18:00:15.924364 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:15.924332    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" event={"ID":"89bb8811-92c7-433d-a015-985f290957db","Type":"ContainerStarted","Data":"8a650ebca1eb62577a19a7906fd48e97c124884dfdbabd5288420899b47aaa74"}
Jul 17 18:00:16.931570 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:16.931537    2572 generic.go:356] "Generic (PLEG): container finished" podID="89bb8811-92c7-433d-a015-985f290957db" containerID="8a650ebca1eb62577a19a7906fd48e97c124884dfdbabd5288420899b47aaa74" exitCode=7
Jul 17 18:00:16.932097 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:16.931610    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" event={"ID":"89bb8811-92c7-433d-a015-985f290957db","Type":"ContainerDied","Data":"8a650ebca1eb62577a19a7906fd48e97c124884dfdbabd5288420899b47aaa74"}
Jul 17 18:00:16.932097 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:16.931690    2572 scope.go:117] "RemoveContainer" containerID="8a7b96e6ab6261960b1b0c917b4d5f1bb296e9322d5be8c6413725ed0e02019e"
Jul 17 18:00:16.932097 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:16.931964    2572 scope.go:117] "RemoveContainer" containerID="8a650ebca1eb62577a19a7906fd48e97c124884dfdbabd5288420899b47aaa74"
Jul 17 18:00:16.932263 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:16.932237    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf_odh-ai-gateway-infra(89bb8811-92c7-433d-a015-985f290957db)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" podUID="89bb8811-92c7-433d-a015-985f290957db"
Jul 17 18:00:16.968452 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:16.968411    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"]
Jul 17 18:00:18.072538 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:18.072511    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"
Jul 17 18:00:18.945507 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:18.945460    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"
Jul 17 18:00:18.945731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:18.945508    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf" event={"ID":"89bb8811-92c7-433d-a015-985f290957db","Type":"ContainerDied","Data":"ae654c2d0445ebecad32356ed3b50a24af5d63c1743c3a2d1ae20764f1a9e11e"}
Jul 17 18:00:18.945731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:18.945583    2572 scope.go:117] "RemoveContainer" containerID="8a650ebca1eb62577a19a7906fd48e97c124884dfdbabd5288420899b47aaa74"
Jul 17 18:00:18.969748 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:18.969708    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"]
Jul 17 18:00:18.983877 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:18.983848    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59516a0f-29738520-dt2lf"]
Jul 17 18:00:19.186644 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.186605    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"]
Jul 17 18:00:19.187028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.186979    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89bb8811-92c7-433d-a015-985f290957db" containerName="cleanup"
Jul 17 18:00:19.187028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.186996    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="89bb8811-92c7-433d-a015-985f290957db" containerName="cleanup"
Jul 17 18:00:19.187028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.187008    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89bb8811-92c7-433d-a015-985f290957db" containerName="cleanup"
Jul 17 18:00:19.187028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.187013    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="89bb8811-92c7-433d-a015-985f290957db" containerName="cleanup"
Jul 17 18:00:19.187028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.187020    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89bb8811-92c7-433d-a015-985f290957db" containerName="cleanup"
Jul 17 18:00:19.187028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.187025    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="89bb8811-92c7-433d-a015-985f290957db" containerName="cleanup"
Jul 17 18:00:19.187223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.187100    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="89bb8811-92c7-433d-a015-985f290957db" containerName="cleanup"
Jul 17 18:00:19.187223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.187110    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="89bb8811-92c7-433d-a015-985f290957db" containerName="cleanup"
Jul 17 18:00:19.187223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.187116    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e26efc90-cfa9-48ba-9542-f80d8fb1cdb5" containerName="revoke-keys"
Jul 17 18:00:19.187223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.187122    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="89bb8811-92c7-433d-a015-985f290957db" containerName="cleanup"
Jul 17 18:00:19.191544 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.191522    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"
Jul 17 18:00:19.204368 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.204336    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"]
Jul 17 18:00:19.312580 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.312537    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a210dec-d46b-4891-842b-75a4bba8a042-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-dfljz\" (UID: \"9a210dec-d46b-4891-842b-75a4bba8a042\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"
Jul 17 18:00:19.413450 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.413412    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a210dec-d46b-4891-842b-75a4bba8a042-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-dfljz\" (UID: \"9a210dec-d46b-4891-842b-75a4bba8a042\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"
Jul 17 18:00:19.414122 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.414101    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a210dec-d46b-4891-842b-75a4bba8a042-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-dfljz\" (UID: \"9a210dec-d46b-4891-842b-75a4bba8a042\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"
Jul 17 18:00:19.501759 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.501676    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"
Jul 17 18:00:19.649187 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.649161    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"]
Jul 17 18:00:19.651601 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:00:19.651571    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a210dec_d46b_4891_842b_75a4bba8a042.slice/crio-9c6f4f6452fa711a86905520443e72f186845ce8fb85c511ab41b5294995cc13 WatchSource:0}: Error finding container 9c6f4f6452fa711a86905520443e72f186845ce8fb85c511ab41b5294995cc13: Status 404 returned error can't find the container with id 9c6f4f6452fa711a86905520443e72f186845ce8fb85c511ab41b5294995cc13
Jul 17 18:00:19.952357 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.952314    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" event={"ID":"9a210dec-d46b-4891-842b-75a4bba8a042","Type":"ContainerStarted","Data":"92b80121d6117c9db1fed62f9475f870a412dcf6900f455d65ef84dccc2ae95c"}
Jul 17 18:00:19.952692 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.952407    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" event={"ID":"9a210dec-d46b-4891-842b-75a4bba8a042","Type":"ContainerStarted","Data":"9c6f4f6452fa711a86905520443e72f186845ce8fb85c511ab41b5294995cc13"}
Jul 17 18:00:19.985976 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:19.985905    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" podStartSLOduration=0.985890072 podStartE2EDuration="985.890072ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:00:19.985682276 +0000 UTC m=+2191.991038129" watchObservedRunningTime="2026-07-17 18:00:19.985890072 +0000 UTC m=+2191.991245978"
Jul 17 18:00:20.515514 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:20.515463    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89bb8811-92c7-433d-a015-985f290957db" path="/var/lib/kubelet/pods/89bb8811-92c7-433d-a015-985f290957db/volumes"
Jul 17 18:00:20.960057 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:20.960022    2572 generic.go:356] "Generic (PLEG): container finished" podID="9a210dec-d46b-4891-842b-75a4bba8a042" containerID="92b80121d6117c9db1fed62f9475f870a412dcf6900f455d65ef84dccc2ae95c" exitCode=6
Jul 17 18:00:20.960271 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:20.960112    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" event={"ID":"9a210dec-d46b-4891-842b-75a4bba8a042","Type":"ContainerDied","Data":"92b80121d6117c9db1fed62f9475f870a412dcf6900f455d65ef84dccc2ae95c"}
Jul 17 18:00:20.960271 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:20.960255    2572 scope.go:117] "RemoveContainer" containerID="92b80121d6117c9db1fed62f9475f870a412dcf6900f455d65ef84dccc2ae95c"
Jul 17 18:00:21.509756 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:21.509669    2572 scope.go:117] "RemoveContainer" containerID="ccd3c144d96c42142fa35f5cfb1c53e5e05485c75b9bbbcb1a917c6e6df11aa9"
Jul 17 18:00:21.967461 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:21.967423    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" event={"ID":"0ee4ff7f-32da-43f4-a96f-17a37415629c","Type":"ContainerStarted","Data":"97cb0b322baa9104fa9efe85870c0f9b115d993df39ed64d89d264862caa8071"}
Jul 17 18:00:21.969291 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:21.969258    2572 generic.go:356] "Generic (PLEG): container finished" podID="9a210dec-d46b-4891-842b-75a4bba8a042" containerID="cecee47b2bc49d0b4238c12b7e01dd1a3300fb48b97e9051ab3742bfecc34fe4" exitCode=6
Jul 17 18:00:21.969456 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:21.969329    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" event={"ID":"9a210dec-d46b-4891-842b-75a4bba8a042","Type":"ContainerDied","Data":"cecee47b2bc49d0b4238c12b7e01dd1a3300fb48b97e9051ab3742bfecc34fe4"}
Jul 17 18:00:21.969456 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:21.969398    2572 scope.go:117] "RemoveContainer" containerID="92b80121d6117c9db1fed62f9475f870a412dcf6900f455d65ef84dccc2ae95c"
Jul 17 18:00:21.969669 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:21.969652    2572 scope.go:117] "RemoveContainer" containerID="cecee47b2bc49d0b4238c12b7e01dd1a3300fb48b97e9051ab3742bfecc34fe4"
Jul 17 18:00:21.969864 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:21.969844    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-dfljz_odh-ai-gateway-infra(9a210dec-d46b-4891-842b-75a4bba8a042)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" podUID="9a210dec-d46b-4891-842b-75a4bba8a042"
Jul 17 18:00:21.988816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:21.988763    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" podStartSLOduration=14.98874608 podStartE2EDuration="14.98874608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:00:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:00:21.986851287 +0000 UTC m=+2193.992207144" watchObservedRunningTime="2026-07-17 18:00:21.98874608 +0000 UTC m=+2193.994101933"
Jul 17 18:00:22.977611 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:22.977578    2572 generic.go:356] "Generic (PLEG): container finished" podID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerID="97cb0b322baa9104fa9efe85870c0f9b115d993df39ed64d89d264862caa8071" exitCode=6
Jul 17 18:00:22.978081 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:22.977647    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" event={"ID":"0ee4ff7f-32da-43f4-a96f-17a37415629c","Type":"ContainerDied","Data":"97cb0b322baa9104fa9efe85870c0f9b115d993df39ed64d89d264862caa8071"}
Jul 17 18:00:22.978081 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:22.977706    2572 scope.go:117] "RemoveContainer" containerID="ccd3c144d96c42142fa35f5cfb1c53e5e05485c75b9bbbcb1a917c6e6df11aa9"
Jul 17 18:00:22.978081 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:22.978005    2572 scope.go:117] "RemoveContainer" containerID="97cb0b322baa9104fa9efe85870c0f9b115d993df39ed64d89d264862caa8071"
Jul 17 18:00:22.978257 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:22.978234    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld_odh-ai-gateway-infra(0ee4ff7f-32da-43f4-a96f-17a37415629c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c"
Jul 17 18:00:22.979743 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:22.979722    2572 scope.go:117] "RemoveContainer" containerID="cecee47b2bc49d0b4238c12b7e01dd1a3300fb48b97e9051ab3742bfecc34fe4"
Jul 17 18:00:22.979889 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:22.979872    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-dfljz_odh-ai-gateway-infra(9a210dec-d46b-4891-842b-75a4bba8a042)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" podUID="9a210dec-d46b-4891-842b-75a4bba8a042"
Jul 17 18:00:22.999818 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:22.999784    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"]
Jul 17 18:00:24.130529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:24.130474    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"
Jul 17 18:00:24.255466 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:24.255365    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee4ff7f-32da-43f4-a96f-17a37415629c-maas-api-service-ca\") pod \"0ee4ff7f-32da-43f4-a96f-17a37415629c\" (UID: \"0ee4ff7f-32da-43f4-a96f-17a37415629c\") "
Jul 17 18:00:24.255794 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:24.255772    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ee4ff7f-32da-43f4-a96f-17a37415629c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ee4ff7f-32da-43f4-a96f-17a37415629c" (UID: "0ee4ff7f-32da-43f4-a96f-17a37415629c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:00:24.356131 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:24.356087    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee4ff7f-32da-43f4-a96f-17a37415629c-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:00:24.993777 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:24.993736    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld" event={"ID":"0ee4ff7f-32da-43f4-a96f-17a37415629c","Type":"ContainerDied","Data":"5512e2df2ff5ea1e2681f0220aeb3b393edbba8241b4245fafdd966c426ac60d"}
Jul 17 18:00:24.993777 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:24.993750    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"
Jul 17 18:00:24.994037 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:24.993851    2572 scope.go:117] "RemoveContainer" containerID="97cb0b322baa9104fa9efe85870c0f9b115d993df39ed64d89d264862caa8071"
Jul 17 18:00:25.014713 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:25.014677    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"]
Jul 17 18:00:25.017825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:25.017796    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-c8wld"]
Jul 17 18:00:26.515095 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:26.515060    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" path="/var/lib/kubelet/pods/0ee4ff7f-32da-43f4-a96f-17a37415629c/volumes"
Jul 17 18:00:34.510619 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:34.510589    2572 scope.go:117] "RemoveContainer" containerID="cecee47b2bc49d0b4238c12b7e01dd1a3300fb48b97e9051ab3742bfecc34fe4"
Jul 17 18:00:35.050051 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:35.050018    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" event={"ID":"9a210dec-d46b-4891-842b-75a4bba8a042","Type":"ContainerStarted","Data":"1946a66f3d78dcd676696f804e7ae5cf6e47027ef1b2ffd41f501c5fc121979c"}
Jul 17 18:00:36.057284 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:36.057247    2572 generic.go:356] "Generic (PLEG): container finished" podID="9a210dec-d46b-4891-842b-75a4bba8a042" containerID="1946a66f3d78dcd676696f804e7ae5cf6e47027ef1b2ffd41f501c5fc121979c" exitCode=6
Jul 17 18:00:36.057805 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:36.057320    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" event={"ID":"9a210dec-d46b-4891-842b-75a4bba8a042","Type":"ContainerDied","Data":"1946a66f3d78dcd676696f804e7ae5cf6e47027ef1b2ffd41f501c5fc121979c"}
Jul 17 18:00:36.057805 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:36.057386    2572 scope.go:117] "RemoveContainer" containerID="cecee47b2bc49d0b4238c12b7e01dd1a3300fb48b97e9051ab3742bfecc34fe4"
Jul 17 18:00:36.057805 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:36.057695    2572 scope.go:117] "RemoveContainer" containerID="1946a66f3d78dcd676696f804e7ae5cf6e47027ef1b2ffd41f501c5fc121979c"
Jul 17 18:00:36.057978 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:36.057955    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-dfljz_odh-ai-gateway-infra(9a210dec-d46b-4891-842b-75a4bba8a042)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" podUID="9a210dec-d46b-4891-842b-75a4bba8a042"
Jul 17 18:00:36.085648 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:36.085609    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"]
Jul 17 18:00:37.206907 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:37.206876    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"
Jul 17 18:00:37.368616 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:37.368522    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a210dec-d46b-4891-842b-75a4bba8a042-maas-api-service-ca\") pod \"9a210dec-d46b-4891-842b-75a4bba8a042\" (UID: \"9a210dec-d46b-4891-842b-75a4bba8a042\") "
Jul 17 18:00:37.368889 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:37.368867    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a210dec-d46b-4891-842b-75a4bba8a042-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9a210dec-d46b-4891-842b-75a4bba8a042" (UID: "9a210dec-d46b-4891-842b-75a4bba8a042"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:00:37.469148 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:37.469107    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a210dec-d46b-4891-842b-75a4bba8a042-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:00:38.071755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:38.071728    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"
Jul 17 18:00:38.071755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:38.071742    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz" event={"ID":"9a210dec-d46b-4891-842b-75a4bba8a042","Type":"ContainerDied","Data":"9c6f4f6452fa711a86905520443e72f186845ce8fb85c511ab41b5294995cc13"}
Jul 17 18:00:38.072071 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:38.071804    2572 scope.go:117] "RemoveContainer" containerID="1946a66f3d78dcd676696f804e7ae5cf6e47027ef1b2ffd41f501c5fc121979c"
Jul 17 18:00:38.096388 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:38.096349    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"]
Jul 17 18:00:38.104560 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:38.104531    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dfljz"]
Jul 17 18:00:38.521011 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:38.520926    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" path="/var/lib/kubelet/pods/9a210dec-d46b-4891-842b-75a4bba8a042/volumes"
Jul 17 18:00:46.121640 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.121604    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"]
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.121926    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.121938    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.121945    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.121950    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.121957    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.121962    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.121972    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.121977    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.122035    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.122042    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerName="revoke-keys"
Jul 17 18:00:46.122045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.122050    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" containerName="revoke-keys"
Jul 17 18:00:46.122383 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.122056    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" containerName="revoke-keys"
Jul 17 18:00:46.122383 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.122062    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" containerName="revoke-keys"
Jul 17 18:00:46.126476 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.126452    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"
Jul 17 18:00:46.129348 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.129318    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:00:46.129636 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.129619    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:00:46.134837 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.134811    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"]
Jul 17 18:00:46.248637 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.248590    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5fbc0a-6835-495c-a491-8ce5fe7236f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-bb24n\" (UID: \"cf5fbc0a-6835-495c-a491-8ce5fe7236f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"
Jul 17 18:00:46.349610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.349573    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5fbc0a-6835-495c-a491-8ce5fe7236f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-bb24n\" (UID: \"cf5fbc0a-6835-495c-a491-8ce5fe7236f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"
Jul 17 18:00:46.350229 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.350208    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5fbc0a-6835-495c-a491-8ce5fe7236f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-bb24n\" (UID: \"cf5fbc0a-6835-495c-a491-8ce5fe7236f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"
Jul 17 18:00:46.438331 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.438218    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"
Jul 17 18:00:46.585770 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:46.585739    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"]
Jul 17 18:00:46.588366 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:00:46.588335    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5fbc0a_6835_495c_a491_8ce5fe7236f2.slice/crio-65a0637c86405f327bacea2c7a383e23fa9f41cb764d24d11fa81db35c49bd5d WatchSource:0}: Error finding container 65a0637c86405f327bacea2c7a383e23fa9f41cb764d24d11fa81db35c49bd5d: Status 404 returned error can't find the container with id 65a0637c86405f327bacea2c7a383e23fa9f41cb764d24d11fa81db35c49bd5d
Jul 17 18:00:47.126523 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:47.126395    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" event={"ID":"cf5fbc0a-6835-495c-a491-8ce5fe7236f2","Type":"ContainerStarted","Data":"fc20d93993a31c89d74efd4cdda54f168d651d1a9829a241fbb1288b94601f80"}
Jul 17 18:00:47.126523 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:47.126461    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" event={"ID":"cf5fbc0a-6835-495c-a491-8ce5fe7236f2","Type":"ContainerStarted","Data":"65a0637c86405f327bacea2c7a383e23fa9f41cb764d24d11fa81db35c49bd5d"}
Jul 17 18:00:47.157688 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:47.157623    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" podStartSLOduration=1.15760339 podStartE2EDuration="1.15760339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:00:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:00:47.155292522 +0000 UTC m=+2219.160648379" watchObservedRunningTime="2026-07-17 18:00:47.15760339 +0000 UTC m=+2219.162959337"
Jul 17 18:00:48.133047 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:48.133011    2572 generic.go:356] "Generic (PLEG): container finished" podID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerID="fc20d93993a31c89d74efd4cdda54f168d651d1a9829a241fbb1288b94601f80" exitCode=7
Jul 17 18:00:48.133512 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:48.133098    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" event={"ID":"cf5fbc0a-6835-495c-a491-8ce5fe7236f2","Type":"ContainerDied","Data":"fc20d93993a31c89d74efd4cdda54f168d651d1a9829a241fbb1288b94601f80"}
Jul 17 18:00:48.133512 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:48.133280    2572 scope.go:117] "RemoveContainer" containerID="fc20d93993a31c89d74efd4cdda54f168d651d1a9829a241fbb1288b94601f80"
Jul 17 18:00:49.139742 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:49.139704    2572 generic.go:356] "Generic (PLEG): container finished" podID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerID="dc3f57693bf3edc66cea8c9a32b3ab459f451264f113230a05e2a08a6bbcf5d3" exitCode=7
Jul 17 18:00:49.140175 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:49.139796    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" event={"ID":"cf5fbc0a-6835-495c-a491-8ce5fe7236f2","Type":"ContainerDied","Data":"dc3f57693bf3edc66cea8c9a32b3ab459f451264f113230a05e2a08a6bbcf5d3"}
Jul 17 18:00:49.140175 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:49.139885    2572 scope.go:117] "RemoveContainer" containerID="fc20d93993a31c89d74efd4cdda54f168d651d1a9829a241fbb1288b94601f80"
Jul 17 18:00:49.140175 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:49.140072    2572 scope.go:117] "RemoveContainer" containerID="dc3f57693bf3edc66cea8c9a32b3ab459f451264f113230a05e2a08a6bbcf5d3"
Jul 17 18:00:49.140354 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:49.140330    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-bb24n_odh-ai-gateway-infra(cf5fbc0a-6835-495c-a491-8ce5fe7236f2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2"
Jul 17 18:00:50.146886 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:50.146857    2572 scope.go:117] "RemoveContainer" containerID="dc3f57693bf3edc66cea8c9a32b3ab459f451264f113230a05e2a08a6bbcf5d3"
Jul 17 18:00:50.147324 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:00:50.147033    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-bb24n_odh-ai-gateway-infra(cf5fbc0a-6835-495c-a491-8ce5fe7236f2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2"
Jul 17 18:00:57.665733 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.665689    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"]
Jul 17 18:00:57.666148 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.666061    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" containerName="revoke-keys"
Jul 17 18:00:57.666148 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.666072    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a210dec-d46b-4891-842b-75a4bba8a042" containerName="revoke-keys"
Jul 17 18:00:57.666148 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.666080    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerName="revoke-keys"
Jul 17 18:00:57.666148 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.666086    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerName="revoke-keys"
Jul 17 18:00:57.666284 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.666164    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ee4ff7f-32da-43f4-a96f-17a37415629c" containerName="revoke-keys"
Jul 17 18:00:57.669148 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.669128    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"
Jul 17 18:00:57.677770 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.677741    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"]
Jul 17 18:00:57.745260 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.745217    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2013dc22-4f2f-4e4e-af8e-27a2ac7404b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j\" (UID: \"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"
Jul 17 18:00:57.846389 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.846347    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2013dc22-4f2f-4e4e-af8e-27a2ac7404b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j\" (UID: \"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"
Jul 17 18:00:57.847019 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.846999    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2013dc22-4f2f-4e4e-af8e-27a2ac7404b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j\" (UID: \"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"
Jul 17 18:00:57.981622 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:57.981477    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"
Jul 17 18:00:58.333727 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:58.333694    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"]
Jul 17 18:00:58.335398 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:00:58.335372    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2013dc22_4f2f_4e4e_af8e_27a2ac7404b5.slice/crio-f4c2edf1891b5eec4b8b8226cb28ded0dc2683aee1a0f4b465f096c8219d672d WatchSource:0}: Error finding container f4c2edf1891b5eec4b8b8226cb28ded0dc2683aee1a0f4b465f096c8219d672d: Status 404 returned error can't find the container with id f4c2edf1891b5eec4b8b8226cb28ded0dc2683aee1a0f4b465f096c8219d672d
Jul 17 18:00:59.198043 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:59.198008    2572 generic.go:356] "Generic (PLEG): container finished" podID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerID="c925faca81e79151aeabd25ee756de488b844a29dd83e286a55d570ee9a397b0" exitCode=6
Jul 17 18:00:59.198601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:59.198094    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j" event={"ID":"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5","Type":"ContainerDied","Data":"c925faca81e79151aeabd25ee756de488b844a29dd83e286a55d570ee9a397b0"}
Jul 17 18:00:59.198601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:59.198151    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j" event={"ID":"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5","Type":"ContainerStarted","Data":"f4c2edf1891b5eec4b8b8226cb28ded0dc2683aee1a0f4b465f096c8219d672d"}
Jul 17 18:00:59.198601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:00:59.198368    2572 scope.go:117] "RemoveContainer" containerID="c925faca81e79151aeabd25ee756de488b844a29dd83e286a55d570ee9a397b0"
Jul 17 18:01:00.205771 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:00.205677    2572 generic.go:356] "Generic (PLEG): container finished" podID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerID="c7ff560bb2213afe7639d1f5ef0bf6c7a7454e0c9766c3b0a263dcc2a1ab5687" exitCode=6
Jul 17 18:01:00.206231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:00.205764    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j" event={"ID":"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5","Type":"ContainerDied","Data":"c7ff560bb2213afe7639d1f5ef0bf6c7a7454e0c9766c3b0a263dcc2a1ab5687"}
Jul 17 18:01:00.206231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:00.205831    2572 scope.go:117] "RemoveContainer" containerID="c925faca81e79151aeabd25ee756de488b844a29dd83e286a55d570ee9a397b0"
Jul 17 18:01:00.206231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:00.206005    2572 scope.go:117] "RemoveContainer" containerID="c7ff560bb2213afe7639d1f5ef0bf6c7a7454e0c9766c3b0a263dcc2a1ab5687"
Jul 17 18:01:00.206231 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:00.206217    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j_odh-ai-gateway-infra(2013dc22-4f2f-4e4e-af8e-27a2ac7404b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5"
Jul 17 18:01:01.214011 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:01.213982    2572 scope.go:117] "RemoveContainer" containerID="c7ff560bb2213afe7639d1f5ef0bf6c7a7454e0c9766c3b0a263dcc2a1ab5687"
Jul 17 18:01:01.214548 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:01.214245    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j_odh-ai-gateway-infra(2013dc22-4f2f-4e4e-af8e-27a2ac7404b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5"
Jul 17 18:01:01.510700 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:01.510612    2572 scope.go:117] "RemoveContainer" containerID="dc3f57693bf3edc66cea8c9a32b3ab459f451264f113230a05e2a08a6bbcf5d3"
Jul 17 18:01:02.221207 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:02.221167    2572 generic.go:356] "Generic (PLEG): container finished" podID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerID="98d02c5d318aeaa87e084b13e3983dcc4dc1d8a6f44101bc01686e684c8abadf" exitCode=7
Jul 17 18:01:02.221701 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:02.221240    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" event={"ID":"cf5fbc0a-6835-495c-a491-8ce5fe7236f2","Type":"ContainerDied","Data":"98d02c5d318aeaa87e084b13e3983dcc4dc1d8a6f44101bc01686e684c8abadf"}
Jul 17 18:01:02.221701 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:02.221307    2572 scope.go:117] "RemoveContainer" containerID="dc3f57693bf3edc66cea8c9a32b3ab459f451264f113230a05e2a08a6bbcf5d3"
Jul 17 18:01:02.221701 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:02.221646    2572 scope.go:117] "RemoveContainer" containerID="98d02c5d318aeaa87e084b13e3983dcc4dc1d8a6f44101bc01686e684c8abadf"
Jul 17 18:01:02.221887 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:02.221843    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-bb24n_odh-ai-gateway-infra(cf5fbc0a-6835-495c-a491-8ce5fe7236f2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2"
Jul 17 18:01:03.264607 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:03.264564    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"]
Jul 17 18:01:03.403572 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:03.403545    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"
Jul 17 18:01:03.495186 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:03.495144    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5fbc0a-6835-495c-a491-8ce5fe7236f2-maas-api-service-ca\") pod \"cf5fbc0a-6835-495c-a491-8ce5fe7236f2\" (UID: \"cf5fbc0a-6835-495c-a491-8ce5fe7236f2\") "
Jul 17 18:01:03.495602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:03.495578    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf5fbc0a-6835-495c-a491-8ce5fe7236f2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf5fbc0a-6835-495c-a491-8ce5fe7236f2" (UID: "cf5fbc0a-6835-495c-a491-8ce5fe7236f2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:01:03.596163 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:03.596112    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5fbc0a-6835-495c-a491-8ce5fe7236f2-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:01:04.234940 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:04.234898    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n" event={"ID":"cf5fbc0a-6835-495c-a491-8ce5fe7236f2","Type":"ContainerDied","Data":"65a0637c86405f327bacea2c7a383e23fa9f41cb764d24d11fa81db35c49bd5d"}
Jul 17 18:01:04.235157 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:04.234965    2572 scope.go:117] "RemoveContainer" containerID="98d02c5d318aeaa87e084b13e3983dcc4dc1d8a6f44101bc01686e684c8abadf"
Jul 17 18:01:04.235157 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:04.234910    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"
Jul 17 18:01:04.257901 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:04.257865    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"]
Jul 17 18:01:04.262634 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:04.262603    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-bb24n"]
Jul 17 18:01:04.516057 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:04.515972    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" path="/var/lib/kubelet/pods/cf5fbc0a-6835-495c-a491-8ce5fe7236f2/volumes"
Jul 17 18:01:09.217646 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.217602    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"]
Jul 17 18:01:09.218058 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.217946    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerName="revoke-keys"
Jul 17 18:01:09.218058 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.217958    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerName="revoke-keys"
Jul 17 18:01:09.218058 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.217973    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerName="revoke-keys"
Jul 17 18:01:09.218058 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.217978    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerName="revoke-keys"
Jul 17 18:01:09.218058 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.218039    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerName="revoke-keys"
Jul 17 18:01:09.218058 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.218048    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerName="revoke-keys"
Jul 17 18:01:09.218058 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.218054    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerName="revoke-keys"
Jul 17 18:01:09.222347 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.222322    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"
Jul 17 18:01:09.228740 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.228702    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"]
Jul 17 18:01:09.248086 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.248049    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5411bd-63bf-45e2-b672-7068ccbcacdd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-dr522\" (UID: \"6f5411bd-63bf-45e2-b672-7068ccbcacdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"
Jul 17 18:01:09.348928 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.348893    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5411bd-63bf-45e2-b672-7068ccbcacdd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-dr522\" (UID: \"6f5411bd-63bf-45e2-b672-7068ccbcacdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"
Jul 17 18:01:09.349635 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.349608    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5411bd-63bf-45e2-b672-7068ccbcacdd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-dr522\" (UID: \"6f5411bd-63bf-45e2-b672-7068ccbcacdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"
Jul 17 18:01:09.535397 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.535276    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"
Jul 17 18:01:09.887526 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:09.887479    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"]
Jul 17 18:01:09.889722 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:01:09.889692    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5411bd_63bf_45e2_b672_7068ccbcacdd.slice/crio-031664e467c120cdd101800aa050261f937d7952d22ef335707cedbea6acbfa9 WatchSource:0}: Error finding container 031664e467c120cdd101800aa050261f937d7952d22ef335707cedbea6acbfa9: Status 404 returned error can't find the container with id 031664e467c120cdd101800aa050261f937d7952d22ef335707cedbea6acbfa9
Jul 17 18:01:10.267160 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:10.267124    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" event={"ID":"6f5411bd-63bf-45e2-b672-7068ccbcacdd","Type":"ContainerStarted","Data":"17239d5586a73eefaabeb36460d5a7c0e3fffbc5acf42433f3c6055de4a9ac4f"}
Jul 17 18:01:10.267606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:10.267178    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" event={"ID":"6f5411bd-63bf-45e2-b672-7068ccbcacdd","Type":"ContainerStarted","Data":"031664e467c120cdd101800aa050261f937d7952d22ef335707cedbea6acbfa9"}
Jul 17 18:01:10.285223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:10.285164    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" podStartSLOduration=1.285144244 podStartE2EDuration="1.285144244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:01:10.282621128 +0000 UTC m=+2242.287976984" watchObservedRunningTime="2026-07-17 18:01:10.285144244 +0000 UTC m=+2242.290500098"
Jul 17 18:01:11.274119 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:11.274078    2572 generic.go:356] "Generic (PLEG): container finished" podID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerID="17239d5586a73eefaabeb36460d5a7c0e3fffbc5acf42433f3c6055de4a9ac4f" exitCode=6
Jul 17 18:01:11.274560 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:11.274160    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" event={"ID":"6f5411bd-63bf-45e2-b672-7068ccbcacdd","Type":"ContainerDied","Data":"17239d5586a73eefaabeb36460d5a7c0e3fffbc5acf42433f3c6055de4a9ac4f"}
Jul 17 18:01:11.274560 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:11.274379    2572 scope.go:117] "RemoveContainer" containerID="17239d5586a73eefaabeb36460d5a7c0e3fffbc5acf42433f3c6055de4a9ac4f"
Jul 17 18:01:12.282143 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:12.282043    2572 generic.go:356] "Generic (PLEG): container finished" podID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerID="3022d848705d60b0c2189d5741ca1013b7f4761509329b4aad2081fac6183876" exitCode=6
Jul 17 18:01:12.282143 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:12.282130    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" event={"ID":"6f5411bd-63bf-45e2-b672-7068ccbcacdd","Type":"ContainerDied","Data":"3022d848705d60b0c2189d5741ca1013b7f4761509329b4aad2081fac6183876"}
Jul 17 18:01:12.282647 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:12.282198    2572 scope.go:117] "RemoveContainer" containerID="17239d5586a73eefaabeb36460d5a7c0e3fffbc5acf42433f3c6055de4a9ac4f"
Jul 17 18:01:12.282647 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:12.282305    2572 scope.go:117] "RemoveContainer" containerID="3022d848705d60b0c2189d5741ca1013b7f4761509329b4aad2081fac6183876"
Jul 17 18:01:12.282647 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:12.282552    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-dr522_odh-ai-gateway-infra(6f5411bd-63bf-45e2-b672-7068ccbcacdd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd"
Jul 17 18:01:13.289112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:13.289080    2572 scope.go:117] "RemoveContainer" containerID="3022d848705d60b0c2189d5741ca1013b7f4761509329b4aad2081fac6183876"
Jul 17 18:01:13.289541 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:13.289292    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-dr522_odh-ai-gateway-infra(6f5411bd-63bf-45e2-b672-7068ccbcacdd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd"
Jul 17 18:01:13.509586 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:13.509556    2572 scope.go:117] "RemoveContainer" containerID="c7ff560bb2213afe7639d1f5ef0bf6c7a7454e0c9766c3b0a263dcc2a1ab5687"
Jul 17 18:01:14.296245 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:14.296209    2572 generic.go:356] "Generic (PLEG): container finished" podID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerID="9a8a2beb813cb57a82b77c53a223deaba00aa336afe996b900896b408cfc5849" exitCode=6
Jul 17 18:01:14.296699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:14.296277    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j" event={"ID":"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5","Type":"ContainerDied","Data":"9a8a2beb813cb57a82b77c53a223deaba00aa336afe996b900896b408cfc5849"}
Jul 17 18:01:14.296699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:14.296344    2572 scope.go:117] "RemoveContainer" containerID="c7ff560bb2213afe7639d1f5ef0bf6c7a7454e0c9766c3b0a263dcc2a1ab5687"
Jul 17 18:01:14.296699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:14.296680    2572 scope.go:117] "RemoveContainer" containerID="9a8a2beb813cb57a82b77c53a223deaba00aa336afe996b900896b408cfc5849"
Jul 17 18:01:14.296876 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:14.296857    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j_odh-ai-gateway-infra(2013dc22-4f2f-4e4e-af8e-27a2ac7404b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5"
Jul 17 18:01:15.328152 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:15.328112    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"]
Jul 17 18:01:15.489302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:15.489272    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"
Jul 17 18:01:15.607168 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:15.607059    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2013dc22-4f2f-4e4e-af8e-27a2ac7404b5-maas-api-service-ca\") pod \"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5\" (UID: \"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5\") "
Jul 17 18:01:15.607515 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:15.607458    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2013dc22-4f2f-4e4e-af8e-27a2ac7404b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" (UID: "2013dc22-4f2f-4e4e-af8e-27a2ac7404b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:01:15.709813 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:15.709782    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2013dc22-4f2f-4e4e-af8e-27a2ac7404b5-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:01:16.310572 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:16.310541    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"
Jul 17 18:01:16.310572 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:16.310553    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j" event={"ID":"2013dc22-4f2f-4e4e-af8e-27a2ac7404b5","Type":"ContainerDied","Data":"f4c2edf1891b5eec4b8b8226cb28ded0dc2683aee1a0f4b465f096c8219d672d"}
Jul 17 18:01:16.310854 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:16.310615    2572 scope.go:117] "RemoveContainer" containerID="9a8a2beb813cb57a82b77c53a223deaba00aa336afe996b900896b408cfc5849"
Jul 17 18:01:16.336189 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:16.336152    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"]
Jul 17 18:01:16.342302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:16.342271    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-sh22j"]
Jul 17 18:01:16.515745 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:16.515710    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" path="/var/lib/kubelet/pods/2013dc22-4f2f-4e4e-af8e-27a2ac7404b5/volumes"
Jul 17 18:01:23.509928 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:23.509890    2572 scope.go:117] "RemoveContainer" containerID="3022d848705d60b0c2189d5741ca1013b7f4761509329b4aad2081fac6183876"
Jul 17 18:01:24.359415 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:24.359381    2572 generic.go:356] "Generic (PLEG): container finished" podID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerID="a3a6891dc795464c061a0af2250558b59b7b916610395b9ec6a8965d28ede172" exitCode=6
Jul 17 18:01:24.359666 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:24.359449    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" event={"ID":"6f5411bd-63bf-45e2-b672-7068ccbcacdd","Type":"ContainerDied","Data":"a3a6891dc795464c061a0af2250558b59b7b916610395b9ec6a8965d28ede172"}
Jul 17 18:01:24.359666 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:24.359526    2572 scope.go:117] "RemoveContainer" containerID="3022d848705d60b0c2189d5741ca1013b7f4761509329b4aad2081fac6183876"
Jul 17 18:01:24.359894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:24.359877    2572 scope.go:117] "RemoveContainer" containerID="a3a6891dc795464c061a0af2250558b59b7b916610395b9ec6a8965d28ede172"
Jul 17 18:01:24.360135 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:24.360103    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-dr522_odh-ai-gateway-infra(6f5411bd-63bf-45e2-b672-7068ccbcacdd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd"
Jul 17 18:01:25.389179 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:25.389140    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"]
Jul 17 18:01:25.525413 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:25.525383    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"
Jul 17 18:01:25.597146 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:25.597105    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5411bd-63bf-45e2-b672-7068ccbcacdd-maas-api-service-ca\") pod \"6f5411bd-63bf-45e2-b672-7068ccbcacdd\" (UID: \"6f5411bd-63bf-45e2-b672-7068ccbcacdd\") "
Jul 17 18:01:25.597410 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:25.597385    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f5411bd-63bf-45e2-b672-7068ccbcacdd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f5411bd-63bf-45e2-b672-7068ccbcacdd" (UID: "6f5411bd-63bf-45e2-b672-7068ccbcacdd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:01:25.698277 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:25.698177    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5411bd-63bf-45e2-b672-7068ccbcacdd-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:01:26.374370 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:26.374338    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"
Jul 17 18:01:26.374624 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:26.374377    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522" event={"ID":"6f5411bd-63bf-45e2-b672-7068ccbcacdd","Type":"ContainerDied","Data":"031664e467c120cdd101800aa050261f937d7952d22ef335707cedbea6acbfa9"}
Jul 17 18:01:26.374624 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:26.374473    2572 scope.go:117] "RemoveContainer" containerID="a3a6891dc795464c061a0af2250558b59b7b916610395b9ec6a8965d28ede172"
Jul 17 18:01:26.399859 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:26.399821    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"]
Jul 17 18:01:26.402505 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:26.402463    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-dr522"]
Jul 17 18:01:26.515399 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:26.515362    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" path="/var/lib/kubelet/pods/6f5411bd-63bf-45e2-b672-7068ccbcacdd/volumes"
Jul 17 18:01:36.161581 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161544    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"]
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161883    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161895    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161902    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161908    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161918    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161923    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161934    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161940    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161949    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161955    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161965    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.161970    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5fbc0a-6835-495c-a491-8ce5fe7236f2" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.162022    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.162029    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.162036    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.162042    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2013dc22-4f2f-4e4e-af8e-27a2ac7404b5" containerName="revoke-keys"
Jul 17 18:01:36.162068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.162048    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerName="revoke-keys"
Jul 17 18:01:36.166592 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.166570    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"
Jul 17 18:01:36.169556 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.169525    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:01:36.169556 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.169534    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:01:36.174246 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.174203    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"]
Jul 17 18:01:36.296912 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.296865    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5\" (UID: \"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"
Jul 17 18:01:36.397897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.397855    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5\" (UID: \"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"
Jul 17 18:01:36.398623 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.398596    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5\" (UID: \"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"
Jul 17 18:01:36.480654 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.480546    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"
Jul 17 18:01:36.627005 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:36.626975    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"]
Jul 17 18:01:36.629277 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:01:36.629247    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e0f7e7_7de4_4a8b_8a19_c05ddc8a003e.slice/crio-0ea45a941b470a4cc1ddcd52f5c6d879b5362baa59114145af9e274b3589dfdd WatchSource:0}: Error finding container 0ea45a941b470a4cc1ddcd52f5c6d879b5362baa59114145af9e274b3589dfdd: Status 404 returned error can't find the container with id 0ea45a941b470a4cc1ddcd52f5c6d879b5362baa59114145af9e274b3589dfdd
Jul 17 18:01:37.437598 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:37.437559    2572 generic.go:356] "Generic (PLEG): container finished" podID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerID="dc490421c5466c44d64304a29c221edbb2287b2819c29fb37502fc5bd77ee9e3" exitCode=7
Jul 17 18:01:37.438033 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:37.437647    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5" event={"ID":"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e","Type":"ContainerDied","Data":"dc490421c5466c44d64304a29c221edbb2287b2819c29fb37502fc5bd77ee9e3"}
Jul 17 18:01:37.438033 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:37.437753    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5" event={"ID":"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e","Type":"ContainerStarted","Data":"0ea45a941b470a4cc1ddcd52f5c6d879b5362baa59114145af9e274b3589dfdd"}
Jul 17 18:01:37.438033 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:37.437838    2572 scope.go:117] "RemoveContainer" containerID="dc490421c5466c44d64304a29c221edbb2287b2819c29fb37502fc5bd77ee9e3"
Jul 17 18:01:38.445465 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:38.445422    2572 generic.go:356] "Generic (PLEG): container finished" podID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerID="24c1ce4bd90f6cec47c2f09d2dd6ad9c10814cb8b2c89bd1efad52868f79a8e2" exitCode=7
Jul 17 18:01:38.446029 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:38.445526    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5" event={"ID":"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e","Type":"ContainerDied","Data":"24c1ce4bd90f6cec47c2f09d2dd6ad9c10814cb8b2c89bd1efad52868f79a8e2"}
Jul 17 18:01:38.446029 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:38.445604    2572 scope.go:117] "RemoveContainer" containerID="dc490421c5466c44d64304a29c221edbb2287b2819c29fb37502fc5bd77ee9e3"
Jul 17 18:01:38.446029 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:38.445812    2572 scope.go:117] "RemoveContainer" containerID="24c1ce4bd90f6cec47c2f09d2dd6ad9c10814cb8b2c89bd1efad52868f79a8e2"
Jul 17 18:01:38.446214 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:38.446039    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5_odh-ai-gateway-infra(a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e"
Jul 17 18:01:39.453565 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:39.453536    2572 scope.go:117] "RemoveContainer" containerID="24c1ce4bd90f6cec47c2f09d2dd6ad9c10814cb8b2c89bd1efad52868f79a8e2"
Jul 17 18:01:39.453982 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:39.453720    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5_odh-ai-gateway-infra(a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e"
Jul 17 18:01:47.720395 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:47.720353    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"]
Jul 17 18:01:47.720825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:47.720733    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerName="revoke-keys"
Jul 17 18:01:47.720825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:47.720745    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerName="revoke-keys"
Jul 17 18:01:47.720825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:47.720805    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f5411bd-63bf-45e2-b672-7068ccbcacdd" containerName="revoke-keys"
Jul 17 18:01:47.725109 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:47.725088    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"
Jul 17 18:01:47.736919 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:47.736884    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"]
Jul 17 18:01:47.796690 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:47.796645    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca34339-47c4-4d58-aee8-2ecc4d39c106-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr\" (UID: \"8ca34339-47c4-4d58-aee8-2ecc4d39c106\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"
Jul 17 18:01:47.897412 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:47.897371    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca34339-47c4-4d58-aee8-2ecc4d39c106-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr\" (UID: \"8ca34339-47c4-4d58-aee8-2ecc4d39c106\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"
Jul 17 18:01:47.898077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:47.898051    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca34339-47c4-4d58-aee8-2ecc4d39c106-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr\" (UID: \"8ca34339-47c4-4d58-aee8-2ecc4d39c106\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"
Jul 17 18:01:48.035767 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:48.035670    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"
Jul 17 18:01:48.181979 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:48.181952    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"]
Jul 17 18:01:48.184133 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:01:48.184099    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca34339_47c4_4d58_aee8_2ecc4d39c106.slice/crio-f006f745484711608c5d18f2edfbda171eb7112d91df4e90f48d44583b2594ec WatchSource:0}: Error finding container f006f745484711608c5d18f2edfbda171eb7112d91df4e90f48d44583b2594ec: Status 404 returned error can't find the container with id f006f745484711608c5d18f2edfbda171eb7112d91df4e90f48d44583b2594ec
Jul 17 18:01:48.505033 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:48.504986    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr" event={"ID":"8ca34339-47c4-4d58-aee8-2ecc4d39c106","Type":"ContainerStarted","Data":"f006f745484711608c5d18f2edfbda171eb7112d91df4e90f48d44583b2594ec"}
Jul 17 18:01:49.512025 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:49.511988    2572 generic.go:356] "Generic (PLEG): container finished" podID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerID="de00cbda0d93efd49864c43d325d952cb4ea0a3d15f33c0b13bb3b7fd2bf58b0" exitCode=6
Jul 17 18:01:49.512444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:49.512064    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr" event={"ID":"8ca34339-47c4-4d58-aee8-2ecc4d39c106","Type":"ContainerDied","Data":"de00cbda0d93efd49864c43d325d952cb4ea0a3d15f33c0b13bb3b7fd2bf58b0"}
Jul 17 18:01:49.512444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:49.512240    2572 scope.go:117] "RemoveContainer" containerID="de00cbda0d93efd49864c43d325d952cb4ea0a3d15f33c0b13bb3b7fd2bf58b0"
Jul 17 18:01:50.519134 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:50.519044    2572 generic.go:356] "Generic (PLEG): container finished" podID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerID="e809444cd697d881cdc122e98f1325b38b192578cf48c8e693b3ea2bb170e454" exitCode=6
Jul 17 18:01:50.519623 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:50.519121    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr" event={"ID":"8ca34339-47c4-4d58-aee8-2ecc4d39c106","Type":"ContainerDied","Data":"e809444cd697d881cdc122e98f1325b38b192578cf48c8e693b3ea2bb170e454"}
Jul 17 18:01:50.519623 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:50.519191    2572 scope.go:117] "RemoveContainer" containerID="de00cbda0d93efd49864c43d325d952cb4ea0a3d15f33c0b13bb3b7fd2bf58b0"
Jul 17 18:01:50.519623 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:50.519324    2572 scope.go:117] "RemoveContainer" containerID="e809444cd697d881cdc122e98f1325b38b192578cf48c8e693b3ea2bb170e454"
Jul 17 18:01:50.519623 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:50.519566    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr_odh-ai-gateway-infra(8ca34339-47c4-4d58-aee8-2ecc4d39c106)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106"
Jul 17 18:01:51.510244 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:51.510208    2572 scope.go:117] "RemoveContainer" containerID="24c1ce4bd90f6cec47c2f09d2dd6ad9c10814cb8b2c89bd1efad52868f79a8e2"
Jul 17 18:01:51.527024 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:51.526999    2572 scope.go:117] "RemoveContainer" containerID="e809444cd697d881cdc122e98f1325b38b192578cf48c8e693b3ea2bb170e454"
Jul 17 18:01:51.527331 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:51.527234    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr_odh-ai-gateway-infra(8ca34339-47c4-4d58-aee8-2ecc4d39c106)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106"
Jul 17 18:01:52.534500 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:52.534379    2572 generic.go:356] "Generic (PLEG): container finished" podID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerID="aa8ec7ab8e1b73220d7b85579342701e2eb41fbf12cf14db0383ac633260be91" exitCode=7
Jul 17 18:01:52.534500 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:52.534451    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5" event={"ID":"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e","Type":"ContainerDied","Data":"aa8ec7ab8e1b73220d7b85579342701e2eb41fbf12cf14db0383ac633260be91"}
Jul 17 18:01:52.534981 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:52.534530    2572 scope.go:117] "RemoveContainer" containerID="24c1ce4bd90f6cec47c2f09d2dd6ad9c10814cb8b2c89bd1efad52868f79a8e2"
Jul 17 18:01:52.534981 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:52.534824    2572 scope.go:117] "RemoveContainer" containerID="aa8ec7ab8e1b73220d7b85579342701e2eb41fbf12cf14db0383ac633260be91"
Jul 17 18:01:52.535083 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:01:52.535027    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5_odh-ai-gateway-infra(a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e"
Jul 17 18:01:53.565471 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:53.565418    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"]
Jul 17 18:01:53.716583 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:53.716549    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"
Jul 17 18:01:53.746733 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:53.746697    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e-maas-api-service-ca\") pod \"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e\" (UID: \"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e\") "
Jul 17 18:01:53.747055 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:53.747031    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" (UID: "a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:01:53.847539 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:53.847413    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:01:54.553458 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:54.553424    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"
Jul 17 18:01:54.553645 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:54.553473    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5" event={"ID":"a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e","Type":"ContainerDied","Data":"0ea45a941b470a4cc1ddcd52f5c6d879b5362baa59114145af9e274b3589dfdd"}
Jul 17 18:01:54.553645 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:54.553558    2572 scope.go:117] "RemoveContainer" containerID="aa8ec7ab8e1b73220d7b85579342701e2eb41fbf12cf14db0383ac633260be91"
Jul 17 18:01:54.574177 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:54.574136    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"]
Jul 17 18:01:54.581547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:54.581513    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-rjwj5"]
Jul 17 18:01:56.515818 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:56.515780    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" path="/var/lib/kubelet/pods/a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e/volumes"
Jul 17 18:01:59.281954 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.281910    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"]
Jul 17 18:01:59.282380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.282275    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerName="revoke-keys"
Jul 17 18:01:59.282380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.282287    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerName="revoke-keys"
Jul 17 18:01:59.282380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.282305    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerName="revoke-keys"
Jul 17 18:01:59.282380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.282311    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerName="revoke-keys"
Jul 17 18:01:59.282380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.282366    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerName="revoke-keys"
Jul 17 18:01:59.282380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.282373    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerName="revoke-keys"
Jul 17 18:01:59.286570 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.286542    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"
Jul 17 18:01:59.302657 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.302624    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"]
Jul 17 18:01:59.400542 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.400466    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a898c7-c56e-48c6-aaaf-11075f887dd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-zljvb\" (UID: \"b6a898c7-c56e-48c6-aaaf-11075f887dd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"
Jul 17 18:01:59.501346 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.501302    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a898c7-c56e-48c6-aaaf-11075f887dd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-zljvb\" (UID: \"b6a898c7-c56e-48c6-aaaf-11075f887dd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"
Jul 17 18:01:59.502029 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.501999    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a898c7-c56e-48c6-aaaf-11075f887dd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-zljvb\" (UID: \"b6a898c7-c56e-48c6-aaaf-11075f887dd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"
Jul 17 18:01:59.597062 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.597016    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"
Jul 17 18:01:59.743790 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:01:59.743760    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"]
Jul 17 18:01:59.746056 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:01:59.746018    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a898c7_c56e_48c6_aaaf_11075f887dd8.slice/crio-3dc131178ad1bade41a2551c5e861fe981f9283b4d476246604d3f3079a2ce8c WatchSource:0}: Error finding container 3dc131178ad1bade41a2551c5e861fe981f9283b4d476246604d3f3079a2ce8c: Status 404 returned error can't find the container with id 3dc131178ad1bade41a2551c5e861fe981f9283b4d476246604d3f3079a2ce8c
Jul 17 18:02:00.587196 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:00.587155    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerID="8402d9e65c4f2da923f58756cd87429fa7c2b3338ebfdffeb6517a5191a54c0e" exitCode=6
Jul 17 18:02:00.587644 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:00.587243    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb" event={"ID":"b6a898c7-c56e-48c6-aaaf-11075f887dd8","Type":"ContainerDied","Data":"8402d9e65c4f2da923f58756cd87429fa7c2b3338ebfdffeb6517a5191a54c0e"}
Jul 17 18:02:00.587644 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:00.587297    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb" event={"ID":"b6a898c7-c56e-48c6-aaaf-11075f887dd8","Type":"ContainerStarted","Data":"3dc131178ad1bade41a2551c5e861fe981f9283b4d476246604d3f3079a2ce8c"}
Jul 17 18:02:00.587644 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:00.587393    2572 scope.go:117] "RemoveContainer" containerID="8402d9e65c4f2da923f58756cd87429fa7c2b3338ebfdffeb6517a5191a54c0e"
Jul 17 18:02:01.594727 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:01.594686    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerID="b34a34dee87db63f23f72050a8ea43fc53d4a8937633591bc54293bd18695d56" exitCode=6
Jul 17 18:02:01.595193 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:01.594777    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb" event={"ID":"b6a898c7-c56e-48c6-aaaf-11075f887dd8","Type":"ContainerDied","Data":"b34a34dee87db63f23f72050a8ea43fc53d4a8937633591bc54293bd18695d56"}
Jul 17 18:02:01.595193 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:01.594833    2572 scope.go:117] "RemoveContainer" containerID="8402d9e65c4f2da923f58756cd87429fa7c2b3338ebfdffeb6517a5191a54c0e"
Jul 17 18:02:01.595193 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:01.595043    2572 scope.go:117] "RemoveContainer" containerID="b34a34dee87db63f23f72050a8ea43fc53d4a8937633591bc54293bd18695d56"
Jul 17 18:02:01.595335 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:01.595254    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-zljvb_odh-ai-gateway-infra(b6a898c7-c56e-48c6-aaaf-11075f887dd8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8"
Jul 17 18:02:02.602359 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:02.602325    2572 scope.go:117] "RemoveContainer" containerID="b34a34dee87db63f23f72050a8ea43fc53d4a8937633591bc54293bd18695d56"
Jul 17 18:02:02.602838 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:02.602622    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-zljvb_odh-ai-gateway-infra(b6a898c7-c56e-48c6-aaaf-11075f887dd8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8"
Jul 17 18:02:06.510519 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:06.510457    2572 scope.go:117] "RemoveContainer" containerID="e809444cd697d881cdc122e98f1325b38b192578cf48c8e693b3ea2bb170e454"
Jul 17 18:02:07.631682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:07.631647    2572 generic.go:356] "Generic (PLEG): container finished" podID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerID="09d78a6c0691cee6071292a1a5f85247f1b659b4a950b49e83c12ed5567d001b" exitCode=6
Jul 17 18:02:07.632130 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:07.631720    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr" event={"ID":"8ca34339-47c4-4d58-aee8-2ecc4d39c106","Type":"ContainerDied","Data":"09d78a6c0691cee6071292a1a5f85247f1b659b4a950b49e83c12ed5567d001b"}
Jul 17 18:02:07.632130 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:07.631781    2572 scope.go:117] "RemoveContainer" containerID="e809444cd697d881cdc122e98f1325b38b192578cf48c8e693b3ea2bb170e454"
Jul 17 18:02:07.632130 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:07.632108    2572 scope.go:117] "RemoveContainer" containerID="09d78a6c0691cee6071292a1a5f85247f1b659b4a950b49e83c12ed5567d001b"
Jul 17 18:02:07.632374 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:07.632350    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr_odh-ai-gateway-infra(8ca34339-47c4-4d58-aee8-2ecc4d39c106)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106"
Jul 17 18:02:08.659073 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:08.659026    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"]
Jul 17 18:02:08.802986 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:08.802953    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"
Jul 17 18:02:08.881801 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:08.881763    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca34339-47c4-4d58-aee8-2ecc4d39c106-maas-api-service-ca\") pod \"8ca34339-47c4-4d58-aee8-2ecc4d39c106\" (UID: \"8ca34339-47c4-4d58-aee8-2ecc4d39c106\") "
Jul 17 18:02:08.882201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:08.882172    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ca34339-47c4-4d58-aee8-2ecc4d39c106-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ca34339-47c4-4d58-aee8-2ecc4d39c106" (UID: "8ca34339-47c4-4d58-aee8-2ecc4d39c106"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:02:08.983438 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:08.983332    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca34339-47c4-4d58-aee8-2ecc4d39c106-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:02:09.648347 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:09.648312    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"
Jul 17 18:02:09.648618 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:09.648345    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr" event={"ID":"8ca34339-47c4-4d58-aee8-2ecc4d39c106","Type":"ContainerDied","Data":"f006f745484711608c5d18f2edfbda171eb7112d91df4e90f48d44583b2594ec"}
Jul 17 18:02:09.648618 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:09.648418    2572 scope.go:117] "RemoveContainer" containerID="09d78a6c0691cee6071292a1a5f85247f1b659b4a950b49e83c12ed5567d001b"
Jul 17 18:02:09.687580 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:09.687543    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"]
Jul 17 18:02:09.696916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:09.696874    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-87cpr"]
Jul 17 18:02:10.515993 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:10.515958    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" path="/var/lib/kubelet/pods/8ca34339-47c4-4d58-aee8-2ecc4d39c106/volumes"
Jul 17 18:02:13.510616 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:13.510577    2572 scope.go:117] "RemoveContainer" containerID="b34a34dee87db63f23f72050a8ea43fc53d4a8937633591bc54293bd18695d56"
Jul 17 18:02:14.677099 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:14.677059    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerID="287f0921a2d89d318bc1d0a488ed87f308c53781ebb546a2a66bf7efb241c615" exitCode=6
Jul 17 18:02:14.677553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:14.677130    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb" event={"ID":"b6a898c7-c56e-48c6-aaaf-11075f887dd8","Type":"ContainerDied","Data":"287f0921a2d89d318bc1d0a488ed87f308c53781ebb546a2a66bf7efb241c615"}
Jul 17 18:02:14.677553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:14.677197    2572 scope.go:117] "RemoveContainer" containerID="b34a34dee87db63f23f72050a8ea43fc53d4a8937633591bc54293bd18695d56"
Jul 17 18:02:14.677553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:14.677532    2572 scope.go:117] "RemoveContainer" containerID="287f0921a2d89d318bc1d0a488ed87f308c53781ebb546a2a66bf7efb241c615"
Jul 17 18:02:14.677772 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:14.677748    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-zljvb_odh-ai-gateway-infra(b6a898c7-c56e-48c6-aaaf-11075f887dd8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8"
Jul 17 18:02:15.705218 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:15.705172    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"]
Jul 17 18:02:15.848054 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:15.848026    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"
Jul 17 18:02:15.943930 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:15.943882    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a898c7-c56e-48c6-aaaf-11075f887dd8-maas-api-service-ca\") pod \"b6a898c7-c56e-48c6-aaaf-11075f887dd8\" (UID: \"b6a898c7-c56e-48c6-aaaf-11075f887dd8\") "
Jul 17 18:02:15.944283 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:15.944257    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6a898c7-c56e-48c6-aaaf-11075f887dd8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b6a898c7-c56e-48c6-aaaf-11075f887dd8" (UID: "b6a898c7-c56e-48c6-aaaf-11075f887dd8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:02:16.045159 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:16.045065    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a898c7-c56e-48c6-aaaf-11075f887dd8-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:02:16.690955 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:16.690919    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb" event={"ID":"b6a898c7-c56e-48c6-aaaf-11075f887dd8","Type":"ContainerDied","Data":"3dc131178ad1bade41a2551c5e861fe981f9283b4d476246604d3f3079a2ce8c"}
Jul 17 18:02:16.690955 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:16.690957    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"
Jul 17 18:02:16.691220 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:16.690985    2572 scope.go:117] "RemoveContainer" containerID="287f0921a2d89d318bc1d0a488ed87f308c53781ebb546a2a66bf7efb241c615"
Jul 17 18:02:16.708072 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:16.708039    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"]
Jul 17 18:02:16.714822 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:16.714792    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-zljvb"]
Jul 17 18:02:18.521350 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:18.521308    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" path="/var/lib/kubelet/pods/b6a898c7-c56e-48c6-aaaf-11075f887dd8/volumes"
Jul 17 18:02:23.819362 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819319    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"]
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819675    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819687    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819699    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819706    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819719    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819724    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819735    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819741    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819750    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819755    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819761    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819767    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819774    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819779    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819826    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4e0f7e7-7de4-4a8b-8a19-c05ddc8a003e" containerName="revoke-keys"
Jul 17 18:02:23.819829 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819833    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerName="revoke-keys"
Jul 17 18:02:23.820324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819840    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerName="revoke-keys"
Jul 17 18:02:23.820324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819846    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerName="revoke-keys"
Jul 17 18:02:23.820324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.819851    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerName="revoke-keys"
Jul 17 18:02:23.824562 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.824529    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"
Jul 17 18:02:23.827562 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.827531    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:02:23.827729 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.827566    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:02:23.833851 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.833825    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"]
Jul 17 18:02:23.921685 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:23.921646    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73061474-f3f7-4cf0-b3aa-42f5511e54c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql\" (UID: \"73061474-f3f7-4cf0-b3aa-42f5511e54c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"
Jul 17 18:02:24.023124 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:24.023084    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73061474-f3f7-4cf0-b3aa-42f5511e54c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql\" (UID: \"73061474-f3f7-4cf0-b3aa-42f5511e54c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"
Jul 17 18:02:24.023744 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:24.023725    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73061474-f3f7-4cf0-b3aa-42f5511e54c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql\" (UID: \"73061474-f3f7-4cf0-b3aa-42f5511e54c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"
Jul 17 18:02:24.137268 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:24.137158    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"
Jul 17 18:02:24.285315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:24.285288    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"]
Jul 17 18:02:24.287336 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:02:24.287306    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73061474_f3f7_4cf0_b3aa_42f5511e54c1.slice/crio-e37e9278ba06161657caff0109387b4f5ee048b9d42c780769296b1d1d58f4f9 WatchSource:0}: Error finding container e37e9278ba06161657caff0109387b4f5ee048b9d42c780769296b1d1d58f4f9: Status 404 returned error can't find the container with id e37e9278ba06161657caff0109387b4f5ee048b9d42c780769296b1d1d58f4f9
Jul 17 18:02:24.735383 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:24.735280    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" event={"ID":"73061474-f3f7-4cf0-b3aa-42f5511e54c1","Type":"ContainerStarted","Data":"0a408b3e003fa0a69786f87b71eb12bac8dedcdcf0cbffdd03e2550b4abf558e"}
Jul 17 18:02:24.735383 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:24.735347    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" event={"ID":"73061474-f3f7-4cf0-b3aa-42f5511e54c1","Type":"ContainerStarted","Data":"e37e9278ba06161657caff0109387b4f5ee048b9d42c780769296b1d1d58f4f9"}
Jul 17 18:02:24.781134 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:24.781070    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" podStartSLOduration=1.781053321 podStartE2EDuration="1.781053321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:02:24.779349034 +0000 UTC m=+2316.784704890" watchObservedRunningTime="2026-07-17 18:02:24.781053321 +0000 UTC m=+2316.786409223"
Jul 17 18:02:25.743621 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:25.743588    2572 generic.go:356] "Generic (PLEG): container finished" podID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerID="0a408b3e003fa0a69786f87b71eb12bac8dedcdcf0cbffdd03e2550b4abf558e" exitCode=7
Jul 17 18:02:25.744053 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:25.743674    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" event={"ID":"73061474-f3f7-4cf0-b3aa-42f5511e54c1","Type":"ContainerDied","Data":"0a408b3e003fa0a69786f87b71eb12bac8dedcdcf0cbffdd03e2550b4abf558e"}
Jul 17 18:02:25.744053 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:25.743832    2572 scope.go:117] "RemoveContainer" containerID="0a408b3e003fa0a69786f87b71eb12bac8dedcdcf0cbffdd03e2550b4abf558e"
Jul 17 18:02:26.215025 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.214985    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"]
Jul 17 18:02:26.215408 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.215395    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ca34339-47c4-4d58-aee8-2ecc4d39c106" containerName="revoke-keys"
Jul 17 18:02:26.215448 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.215411    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6a898c7-c56e-48c6-aaaf-11075f887dd8" containerName="revoke-keys"
Jul 17 18:02:26.218579 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.218558    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"
Jul 17 18:02:26.227669 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.227632    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"]
Jul 17 18:02:26.345082 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.345041    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c50f52-1ec1-4f05-b65d-272259ff03e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-ggn44\" (UID: \"59c50f52-1ec1-4f05-b65d-272259ff03e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"
Jul 17 18:02:26.446279 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.446235    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c50f52-1ec1-4f05-b65d-272259ff03e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-ggn44\" (UID: \"59c50f52-1ec1-4f05-b65d-272259ff03e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"
Jul 17 18:02:26.447002 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.446971    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c50f52-1ec1-4f05-b65d-272259ff03e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-ggn44\" (UID: \"59c50f52-1ec1-4f05-b65d-272259ff03e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"
Jul 17 18:02:26.532088 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.531989    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"
Jul 17 18:02:26.682485 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.682458    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"]
Jul 17 18:02:26.684160 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:02:26.684119    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c50f52_1ec1_4f05_b65d_272259ff03e0.slice/crio-d5e40a566d89127a472dc86807836cf27d771034921411c14058559a0a4c6f4f WatchSource:0}: Error finding container d5e40a566d89127a472dc86807836cf27d771034921411c14058559a0a4c6f4f: Status 404 returned error can't find the container with id d5e40a566d89127a472dc86807836cf27d771034921411c14058559a0a4c6f4f
Jul 17 18:02:26.751653 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.751622    2572 generic.go:356] "Generic (PLEG): container finished" podID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerID="4cdafe8029b6c1287e20c2876533468a88f525c92f9e3e32dfad18b12c435425" exitCode=7
Jul 17 18:02:26.752134 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.751710    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" event={"ID":"73061474-f3f7-4cf0-b3aa-42f5511e54c1","Type":"ContainerDied","Data":"4cdafe8029b6c1287e20c2876533468a88f525c92f9e3e32dfad18b12c435425"}
Jul 17 18:02:26.752134 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.751787    2572 scope.go:117] "RemoveContainer" containerID="0a408b3e003fa0a69786f87b71eb12bac8dedcdcf0cbffdd03e2550b4abf558e"
Jul 17 18:02:26.752134 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.751942    2572 scope.go:117] "RemoveContainer" containerID="4cdafe8029b6c1287e20c2876533468a88f525c92f9e3e32dfad18b12c435425"
Jul 17 18:02:26.752288 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:26.752166    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql_odh-ai-gateway-infra(73061474-f3f7-4cf0-b3aa-42f5511e54c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1"
Jul 17 18:02:26.753090 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:26.753074    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" event={"ID":"59c50f52-1ec1-4f05-b65d-272259ff03e0","Type":"ContainerStarted","Data":"d5e40a566d89127a472dc86807836cf27d771034921411c14058559a0a4c6f4f"}
Jul 17 18:02:27.760071 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:27.759983    2572 scope.go:117] "RemoveContainer" containerID="4cdafe8029b6c1287e20c2876533468a88f525c92f9e3e32dfad18b12c435425"
Jul 17 18:02:27.760510 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:27.760201    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql_odh-ai-gateway-infra(73061474-f3f7-4cf0-b3aa-42f5511e54c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1"
Jul 17 18:02:27.761415 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:27.761389    2572 generic.go:356] "Generic (PLEG): container finished" podID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerID="ab5938ed229f3ee9825368160ea5ab5800221db59adb8b19efdc83ab9f3528de" exitCode=6
Jul 17 18:02:27.761560 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:27.761424    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" event={"ID":"59c50f52-1ec1-4f05-b65d-272259ff03e0","Type":"ContainerDied","Data":"ab5938ed229f3ee9825368160ea5ab5800221db59adb8b19efdc83ab9f3528de"}
Jul 17 18:02:27.761711 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:27.761691    2572 scope.go:117] "RemoveContainer" containerID="ab5938ed229f3ee9825368160ea5ab5800221db59adb8b19efdc83ab9f3528de"
Jul 17 18:02:28.771065 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:28.770966    2572 generic.go:356] "Generic (PLEG): container finished" podID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerID="db2ad7388cce08b5d81ea505304005e3f3c1b2155542a96fd8df29d2f99df2db" exitCode=6
Jul 17 18:02:28.771065 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:28.771017    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" event={"ID":"59c50f52-1ec1-4f05-b65d-272259ff03e0","Type":"ContainerDied","Data":"db2ad7388cce08b5d81ea505304005e3f3c1b2155542a96fd8df29d2f99df2db"}
Jul 17 18:02:28.771582 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:28.771091    2572 scope.go:117] "RemoveContainer" containerID="ab5938ed229f3ee9825368160ea5ab5800221db59adb8b19efdc83ab9f3528de"
Jul 17 18:02:28.771582 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:28.771309    2572 scope.go:117] "RemoveContainer" containerID="db2ad7388cce08b5d81ea505304005e3f3c1b2155542a96fd8df29d2f99df2db"
Jul 17 18:02:28.771582 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:28.771547    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-ggn44_odh-ai-gateway-infra(59c50f52-1ec1-4f05-b65d-272259ff03e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0"
Jul 17 18:02:29.778180 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:29.778150    2572 scope.go:117] "RemoveContainer" containerID="db2ad7388cce08b5d81ea505304005e3f3c1b2155542a96fd8df29d2f99df2db"
Jul 17 18:02:29.778652 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:29.778351    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-ggn44_odh-ai-gateway-infra(59c50f52-1ec1-4f05-b65d-272259ff03e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0"
Jul 17 18:02:38.512897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:38.512855    2572 scope.go:117] "RemoveContainer" containerID="4cdafe8029b6c1287e20c2876533468a88f525c92f9e3e32dfad18b12c435425"
Jul 17 18:02:39.834683 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:39.834645    2572 generic.go:356] "Generic (PLEG): container finished" podID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerID="af3aa1ed5e8f5a2059784044440f82c159723bb1d6f2fb81b0715fe8affd2e6e" exitCode=7
Jul 17 18:02:39.835233 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:39.834718    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" event={"ID":"73061474-f3f7-4cf0-b3aa-42f5511e54c1","Type":"ContainerDied","Data":"af3aa1ed5e8f5a2059784044440f82c159723bb1d6f2fb81b0715fe8affd2e6e"}
Jul 17 18:02:39.835233 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:39.834783    2572 scope.go:117] "RemoveContainer" containerID="4cdafe8029b6c1287e20c2876533468a88f525c92f9e3e32dfad18b12c435425"
Jul 17 18:02:39.835233 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:39.835162    2572 scope.go:117] "RemoveContainer" containerID="af3aa1ed5e8f5a2059784044440f82c159723bb1d6f2fb81b0715fe8affd2e6e"
Jul 17 18:02:39.835438 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:39.835413    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql_odh-ai-gateway-infra(73061474-f3f7-4cf0-b3aa-42f5511e54c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1"
Jul 17 18:02:40.875254 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:40.875167    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"]
Jul 17 18:02:41.020522 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:41.020480    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"
Jul 17 18:02:41.081449 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:41.081408    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73061474-f3f7-4cf0-b3aa-42f5511e54c1-maas-api-service-ca\") pod \"73061474-f3f7-4cf0-b3aa-42f5511e54c1\" (UID: \"73061474-f3f7-4cf0-b3aa-42f5511e54c1\") "
Jul 17 18:02:41.081846 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:41.081823    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73061474-f3f7-4cf0-b3aa-42f5511e54c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73061474-f3f7-4cf0-b3aa-42f5511e54c1" (UID: "73061474-f3f7-4cf0-b3aa-42f5511e54c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:02:41.182882 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:41.182777    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73061474-f3f7-4cf0-b3aa-42f5511e54c1-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:02:41.848661 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:41.848628    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"
Jul 17 18:02:41.848856 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:41.848627    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql" event={"ID":"73061474-f3f7-4cf0-b3aa-42f5511e54c1","Type":"ContainerDied","Data":"e37e9278ba06161657caff0109387b4f5ee048b9d42c780769296b1d1d58f4f9"}
Jul 17 18:02:41.848856 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:41.848760    2572 scope.go:117] "RemoveContainer" containerID="af3aa1ed5e8f5a2059784044440f82c159723bb1d6f2fb81b0715fe8affd2e6e"
Jul 17 18:02:41.877650 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:41.877612    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"]
Jul 17 18:02:41.879700 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:41.879678    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-llpql"]
Jul 17 18:02:42.515257 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:42.515223    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" path="/var/lib/kubelet/pods/73061474-f3f7-4cf0-b3aa-42f5511e54c1/volumes"
Jul 17 18:02:43.510751 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:43.510703    2572 scope.go:117] "RemoveContainer" containerID="db2ad7388cce08b5d81ea505304005e3f3c1b2155542a96fd8df29d2f99df2db"
Jul 17 18:02:43.863001 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:43.862967    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" event={"ID":"59c50f52-1ec1-4f05-b65d-272259ff03e0","Type":"ContainerStarted","Data":"79e26d3ab6622ede82ad6a6fe2a702592053c1f21acc7dd800bc06ede7048c13"}
Jul 17 18:02:43.899177 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:43.899108    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" podStartSLOduration=17.899090623 podStartE2EDuration="17.899090623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:02:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:02:43.897715033 +0000 UTC m=+2335.903070889" watchObservedRunningTime="2026-07-17 18:02:43.899090623 +0000 UTC m=+2335.904446478"
Jul 17 18:02:44.870975 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:44.870941    2572 generic.go:356] "Generic (PLEG): container finished" podID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerID="79e26d3ab6622ede82ad6a6fe2a702592053c1f21acc7dd800bc06ede7048c13" exitCode=6
Jul 17 18:02:44.871415 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:44.870992    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" event={"ID":"59c50f52-1ec1-4f05-b65d-272259ff03e0","Type":"ContainerDied","Data":"79e26d3ab6622ede82ad6a6fe2a702592053c1f21acc7dd800bc06ede7048c13"}
Jul 17 18:02:44.871415 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:44.871042    2572 scope.go:117] "RemoveContainer" containerID="db2ad7388cce08b5d81ea505304005e3f3c1b2155542a96fd8df29d2f99df2db"
Jul 17 18:02:44.871415 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:44.871376    2572 scope.go:117] "RemoveContainer" containerID="79e26d3ab6622ede82ad6a6fe2a702592053c1f21acc7dd800bc06ede7048c13"
Jul 17 18:02:44.871627 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:44.871605    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-ggn44_odh-ai-gateway-infra(59c50f52-1ec1-4f05-b65d-272259ff03e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0"
Jul 17 18:02:44.920095 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:44.920060    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"]
Jul 17 18:02:46.024313 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:46.024275    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"
Jul 17 18:02:46.125169 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:46.125127    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c50f52-1ec1-4f05-b65d-272259ff03e0-maas-api-service-ca\") pod \"59c50f52-1ec1-4f05-b65d-272259ff03e0\" (UID: \"59c50f52-1ec1-4f05-b65d-272259ff03e0\") "
Jul 17 18:02:46.125536 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:46.125484    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59c50f52-1ec1-4f05-b65d-272259ff03e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "59c50f52-1ec1-4f05-b65d-272259ff03e0" (UID: "59c50f52-1ec1-4f05-b65d-272259ff03e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:02:46.226668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:46.226567    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c50f52-1ec1-4f05-b65d-272259ff03e0-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:02:46.884315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:46.884286    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"
Jul 17 18:02:46.884538 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:46.884324    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44" event={"ID":"59c50f52-1ec1-4f05-b65d-272259ff03e0","Type":"ContainerDied","Data":"d5e40a566d89127a472dc86807836cf27d771034921411c14058559a0a4c6f4f"}
Jul 17 18:02:46.884538 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:46.884391    2572 scope.go:117] "RemoveContainer" containerID="79e26d3ab6622ede82ad6a6fe2a702592053c1f21acc7dd800bc06ede7048c13"
Jul 17 18:02:46.904450 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:46.904415    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"]
Jul 17 18:02:46.909632 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:46.909599    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ggn44"]
Jul 17 18:02:47.766973 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.766936    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"]
Jul 17 18:02:47.767472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767432    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerName="revoke-keys"
Jul 17 18:02:47.767472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767450    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerName="revoke-keys"
Jul 17 18:02:47.767472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767470    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767479    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767514    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767524    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767537    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767545    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767629    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767643    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767655    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767663    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerName="revoke-keys"
Jul 17 18:02:47.767682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.767672    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerName="revoke-keys"
Jul 17 18:02:47.771999 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.771971    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"
Jul 17 18:02:47.774990 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.774968    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:02:47.775285 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.775272    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:02:47.780250 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.779713    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"]
Jul 17 18:02:47.840923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.840878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49aea314-c073-419f-b7c8-fd1834ed05b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz\" (UID: \"49aea314-c073-419f-b7c8-fd1834ed05b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"
Jul 17 18:02:47.942222 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.942180    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49aea314-c073-419f-b7c8-fd1834ed05b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz\" (UID: \"49aea314-c073-419f-b7c8-fd1834ed05b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"
Jul 17 18:02:47.942903 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:47.942882    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49aea314-c073-419f-b7c8-fd1834ed05b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz\" (UID: \"49aea314-c073-419f-b7c8-fd1834ed05b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"
Jul 17 18:02:48.084103 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:48.084063    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"
Jul 17 18:02:48.234757 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:48.234728    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"]
Jul 17 18:02:48.236894 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:02:48.236866    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49aea314_c073_419f_b7c8_fd1834ed05b5.slice/crio-943874bbaa878e8c6ae324b872fb79c2976c207ff927ddd24485c415f9d8bed9 WatchSource:0}: Error finding container 943874bbaa878e8c6ae324b872fb79c2976c207ff927ddd24485c415f9d8bed9: Status 404 returned error can't find the container with id 943874bbaa878e8c6ae324b872fb79c2976c207ff927ddd24485c415f9d8bed9
Jul 17 18:02:48.516028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:48.515996    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" path="/var/lib/kubelet/pods/59c50f52-1ec1-4f05-b65d-272259ff03e0/volumes"
Jul 17 18:02:48.898327 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:48.898291    2572 generic.go:356] "Generic (PLEG): container finished" podID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerID="e0d3b70498674ed3b7fca9d6db5dee91f9984850c82a3aa569a356b92c5ae0d1" exitCode=6
Jul 17 18:02:48.898820 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:48.898381    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" event={"ID":"49aea314-c073-419f-b7c8-fd1834ed05b5","Type":"ContainerDied","Data":"e0d3b70498674ed3b7fca9d6db5dee91f9984850c82a3aa569a356b92c5ae0d1"}
Jul 17 18:02:48.898820 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:48.898450    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" event={"ID":"49aea314-c073-419f-b7c8-fd1834ed05b5","Type":"ContainerStarted","Data":"943874bbaa878e8c6ae324b872fb79c2976c207ff927ddd24485c415f9d8bed9"}
Jul 17 18:02:48.898820 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:48.898578    2572 scope.go:117] "RemoveContainer" containerID="e0d3b70498674ed3b7fca9d6db5dee91f9984850c82a3aa569a356b92c5ae0d1"
Jul 17 18:02:49.362539 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.362484    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"]
Jul 17 18:02:49.362860 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.362846    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerName="revoke-keys"
Jul 17 18:02:49.362860 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.362861    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerName="revoke-keys"
Jul 17 18:02:49.362960 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.362875    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerName="revoke-keys"
Jul 17 18:02:49.362960 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.362881    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="73061474-f3f7-4cf0-b3aa-42f5511e54c1" containerName="revoke-keys"
Jul 17 18:02:49.362960 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.362945    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c50f52-1ec1-4f05-b65d-272259ff03e0" containerName="revoke-keys"
Jul 17 18:02:49.365941 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.365921    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"
Jul 17 18:02:49.378127 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.378095    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"]
Jul 17 18:02:49.455629 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.455519    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ad2cc-5455-4c67-8de7-cd22d2b035e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-qknmr\" (UID: \"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"
Jul 17 18:02:49.556127 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.556077    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ad2cc-5455-4c67-8de7-cd22d2b035e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-qknmr\" (UID: \"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"
Jul 17 18:02:49.556772 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.556749    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ad2cc-5455-4c67-8de7-cd22d2b035e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-qknmr\" (UID: \"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"
Jul 17 18:02:49.676847 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.676807    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"
Jul 17 18:02:49.832638 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.832604    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"]
Jul 17 18:02:49.834986 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:02:49.834954    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2ad2cc_5455_4c67_8de7_cd22d2b035e8.slice/crio-a51bcacec82f64101eb7b52886f3a72b55b3c27ea065a863f6d5a6127ce1b19f WatchSource:0}: Error finding container a51bcacec82f64101eb7b52886f3a72b55b3c27ea065a863f6d5a6127ce1b19f: Status 404 returned error can't find the container with id a51bcacec82f64101eb7b52886f3a72b55b3c27ea065a863f6d5a6127ce1b19f
Jul 17 18:02:49.905147 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.905115    2572 generic.go:356] "Generic (PLEG): container finished" podID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerID="c952423de2474dd55f4dc57b90d2f940428cf01dd2937cc732b66dc8b62e8540" exitCode=6
Jul 17 18:02:49.905728 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.905209    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" event={"ID":"49aea314-c073-419f-b7c8-fd1834ed05b5","Type":"ContainerDied","Data":"c952423de2474dd55f4dc57b90d2f940428cf01dd2937cc732b66dc8b62e8540"}
Jul 17 18:02:49.905728 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.905275    2572 scope.go:117] "RemoveContainer" containerID="e0d3b70498674ed3b7fca9d6db5dee91f9984850c82a3aa569a356b92c5ae0d1"
Jul 17 18:02:49.905728 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.905452    2572 scope.go:117] "RemoveContainer" containerID="c952423de2474dd55f4dc57b90d2f940428cf01dd2937cc732b66dc8b62e8540"
Jul 17 18:02:49.905869 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:49.905723    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz_odh-ai-gateway-infra(49aea314-c073-419f-b7c8-fd1834ed05b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5"
Jul 17 18:02:49.906694 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:49.906668    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" event={"ID":"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8","Type":"ContainerStarted","Data":"a51bcacec82f64101eb7b52886f3a72b55b3c27ea065a863f6d5a6127ce1b19f"}
Jul 17 18:02:50.913158 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:50.913125    2572 scope.go:117] "RemoveContainer" containerID="c952423de2474dd55f4dc57b90d2f940428cf01dd2937cc732b66dc8b62e8540"
Jul 17 18:02:50.913698 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:50.913337    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz_odh-ai-gateway-infra(49aea314-c073-419f-b7c8-fd1834ed05b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5"
Jul 17 18:02:50.914538 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:50.914518    2572 generic.go:356] "Generic (PLEG): container finished" podID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerID="d5b08fd3970bab20fb30e43969c0155e2d9d22dad84bda0ddd9662515df6e3ba" exitCode=6
Jul 17 18:02:50.914603 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:50.914576    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" event={"ID":"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8","Type":"ContainerDied","Data":"d5b08fd3970bab20fb30e43969c0155e2d9d22dad84bda0ddd9662515df6e3ba"}
Jul 17 18:02:50.914748 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:50.914734    2572 scope.go:117] "RemoveContainer" containerID="d5b08fd3970bab20fb30e43969c0155e2d9d22dad84bda0ddd9662515df6e3ba"
Jul 17 18:02:51.921746 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:51.921642    2572 generic.go:356] "Generic (PLEG): container finished" podID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerID="39ad6a22f68d9b8e89ac04f188e5f9cf6a58b1c6d6712007763796d13df6aa03" exitCode=6
Jul 17 18:02:51.922223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:51.921737    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" event={"ID":"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8","Type":"ContainerDied","Data":"39ad6a22f68d9b8e89ac04f188e5f9cf6a58b1c6d6712007763796d13df6aa03"}
Jul 17 18:02:51.922223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:51.921809    2572 scope.go:117] "RemoveContainer" containerID="d5b08fd3970bab20fb30e43969c0155e2d9d22dad84bda0ddd9662515df6e3ba"
Jul 17 18:02:51.922223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:51.921958    2572 scope.go:117] "RemoveContainer" containerID="39ad6a22f68d9b8e89ac04f188e5f9cf6a58b1c6d6712007763796d13df6aa03"
Jul 17 18:02:51.922361 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:51.922244    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-qknmr_odh-ai-gateway-infra(9e2ad2cc-5455-4c67-8de7-cd22d2b035e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8"
Jul 17 18:02:52.929776 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:02:52.929738    2572 scope.go:117] "RemoveContainer" containerID="39ad6a22f68d9b8e89ac04f188e5f9cf6a58b1c6d6712007763796d13df6aa03"
Jul 17 18:02:52.930206 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:02:52.929964    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-qknmr_odh-ai-gateway-infra(9e2ad2cc-5455-4c67-8de7-cd22d2b035e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8"
Jul 17 18:03:02.510204 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:02.510164    2572 scope.go:117] "RemoveContainer" containerID="c952423de2474dd55f4dc57b90d2f940428cf01dd2937cc732b66dc8b62e8540"
Jul 17 18:03:02.984109 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:02.984068    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" event={"ID":"49aea314-c073-419f-b7c8-fd1834ed05b5","Type":"ContainerStarted","Data":"6b1112b4568555a02931e29c9f63468b98bb53da37e2fca349fb2bc461c8ab10"}
Jul 17 18:03:03.003887 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:03.003823    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" podStartSLOduration=16.00380582 podStartE2EDuration="16.00380582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:02:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:03.002230013 +0000 UTC m=+2355.007585880" watchObservedRunningTime="2026-07-17 18:03:03.00380582 +0000 UTC m=+2355.009161674"
Jul 17 18:03:03.991622 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:03.991582    2572 generic.go:356] "Generic (PLEG): container finished" podID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerID="6b1112b4568555a02931e29c9f63468b98bb53da37e2fca349fb2bc461c8ab10" exitCode=6
Jul 17 18:03:03.992101 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:03.991657    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" event={"ID":"49aea314-c073-419f-b7c8-fd1834ed05b5","Type":"ContainerDied","Data":"6b1112b4568555a02931e29c9f63468b98bb53da37e2fca349fb2bc461c8ab10"}
Jul 17 18:03:03.992101 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:03.991716    2572 scope.go:117] "RemoveContainer" containerID="c952423de2474dd55f4dc57b90d2f940428cf01dd2937cc732b66dc8b62e8540"
Jul 17 18:03:03.992101 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:03.992015    2572 scope.go:117] "RemoveContainer" containerID="6b1112b4568555a02931e29c9f63468b98bb53da37e2fca349fb2bc461c8ab10"
Jul 17 18:03:03.992252 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:03.992231    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz_odh-ai-gateway-infra(49aea314-c073-419f-b7c8-fd1834ed05b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5"
Jul 17 18:03:04.017718 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:04.017682    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"]
Jul 17 18:03:05.143809 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:05.143772    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"
Jul 17 18:03:05.301942 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:05.301846    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49aea314-c073-419f-b7c8-fd1834ed05b5-maas-api-service-ca\") pod \"49aea314-c073-419f-b7c8-fd1834ed05b5\" (UID: \"49aea314-c073-419f-b7c8-fd1834ed05b5\") "
Jul 17 18:03:05.302248 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:05.302223    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49aea314-c073-419f-b7c8-fd1834ed05b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "49aea314-c073-419f-b7c8-fd1834ed05b5" (UID: "49aea314-c073-419f-b7c8-fd1834ed05b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:03:05.402687 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:05.402632    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49aea314-c073-419f-b7c8-fd1834ed05b5-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:03:06.009226 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:06.009195    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"
Jul 17 18:03:06.009435 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:06.009194    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz" event={"ID":"49aea314-c073-419f-b7c8-fd1834ed05b5","Type":"ContainerDied","Data":"943874bbaa878e8c6ae324b872fb79c2976c207ff927ddd24485c415f9d8bed9"}
Jul 17 18:03:06.009435 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:06.009342    2572 scope.go:117] "RemoveContainer" containerID="6b1112b4568555a02931e29c9f63468b98bb53da37e2fca349fb2bc461c8ab10"
Jul 17 18:03:06.032562 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:06.032521    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"]
Jul 17 18:03:06.040740 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:06.040700    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ps8pz"]
Jul 17 18:03:06.516590 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:06.516555    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" path="/var/lib/kubelet/pods/49aea314-c073-419f-b7c8-fd1834ed05b5/volumes"
Jul 17 18:03:08.513096 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:08.513067    2572 scope.go:117] "RemoveContainer" containerID="39ad6a22f68d9b8e89ac04f188e5f9cf6a58b1c6d6712007763796d13df6aa03"
Jul 17 18:03:09.034212 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:09.034169    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" event={"ID":"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8","Type":"ContainerStarted","Data":"6c8c8af154ab4c96b6ff5e2907a2700d83876eef49ff2e33353b1a61080e2080"}
Jul 17 18:03:09.052782 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:09.052723    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" podStartSLOduration=20.052706643 podStartE2EDuration="20.052706643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:02:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:09.051887576 +0000 UTC m=+2361.057243434" watchObservedRunningTime="2026-07-17 18:03:09.052706643 +0000 UTC m=+2361.058062497"
Jul 17 18:03:10.041375 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:10.041337    2572 generic.go:356] "Generic (PLEG): container finished" podID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerID="6c8c8af154ab4c96b6ff5e2907a2700d83876eef49ff2e33353b1a61080e2080" exitCode=6
Jul 17 18:03:10.041874 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:10.041413    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" event={"ID":"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8","Type":"ContainerDied","Data":"6c8c8af154ab4c96b6ff5e2907a2700d83876eef49ff2e33353b1a61080e2080"}
Jul 17 18:03:10.041874 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:10.041474    2572 scope.go:117] "RemoveContainer" containerID="39ad6a22f68d9b8e89ac04f188e5f9cf6a58b1c6d6712007763796d13df6aa03"
Jul 17 18:03:10.041874 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:10.041784    2572 scope.go:117] "RemoveContainer" containerID="6c8c8af154ab4c96b6ff5e2907a2700d83876eef49ff2e33353b1a61080e2080"
Jul 17 18:03:10.042010 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:10.041979    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-qknmr_odh-ai-gateway-infra(9e2ad2cc-5455-4c67-8de7-cd22d2b035e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8"
Jul 17 18:03:10.067833 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:10.067799    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"]
Jul 17 18:03:11.188129 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:11.188103    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"
Jul 17 18:03:11.354168 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:11.354134    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ad2cc-5455-4c67-8de7-cd22d2b035e8-maas-api-service-ca\") pod \"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8\" (UID: \"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8\") "
Jul 17 18:03:11.354600 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:11.354574    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e2ad2cc-5455-4c67-8de7-cd22d2b035e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" (UID: "9e2ad2cc-5455-4c67-8de7-cd22d2b035e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:03:11.455771 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:11.455735    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ad2cc-5455-4c67-8de7-cd22d2b035e8-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:03:12.055130 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:12.055098    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"
Jul 17 18:03:12.055130 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:12.055114    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr" event={"ID":"9e2ad2cc-5455-4c67-8de7-cd22d2b035e8","Type":"ContainerDied","Data":"a51bcacec82f64101eb7b52886f3a72b55b3c27ea065a863f6d5a6127ce1b19f"}
Jul 17 18:03:12.055396 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:12.055177    2572 scope.go:117] "RemoveContainer" containerID="6c8c8af154ab4c96b6ff5e2907a2700d83876eef49ff2e33353b1a61080e2080"
Jul 17 18:03:12.087145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:12.087110    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"]
Jul 17 18:03:12.094297 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:12.094266    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qknmr"]
Jul 17 18:03:12.515213 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:12.515175    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" path="/var/lib/kubelet/pods/9e2ad2cc-5455-4c67-8de7-cd22d2b035e8/volumes"
Jul 17 18:03:13.911778 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.911738    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"]
Jul 17 18:03:13.912313 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912282    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerName="revoke-keys"
Jul 17 18:03:13.912313 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912303    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerName="revoke-keys"
Jul 17 18:03:13.912439 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912323    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerName="revoke-keys"
Jul 17 18:03:13.912439 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912332    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerName="revoke-keys"
Jul 17 18:03:13.912439 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912346    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerName="revoke-keys"
Jul 17 18:03:13.912439 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912356    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerName="revoke-keys"
Jul 17 18:03:13.912439 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912382    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerName="revoke-keys"
Jul 17 18:03:13.912439 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912391    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerName="revoke-keys"
Jul 17 18:03:13.912841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912473    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerName="revoke-keys"
Jul 17 18:03:13.912841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912484    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerName="revoke-keys"
Jul 17 18:03:13.912841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912529    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerName="revoke-keys"
Jul 17 18:03:13.912841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912537    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerName="revoke-keys"
Jul 17 18:03:13.912841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.912546    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerName="revoke-keys"
Jul 17 18:03:13.916888 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.916862    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"
Jul 17 18:03:13.922773 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.922744    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:03:13.924337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.924308    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:03:13.924916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:13.924550    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"]
Jul 17 18:03:14.078553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:14.078481    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22\" (UID: \"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"
Jul 17 18:03:14.179910 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:14.179805    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22\" (UID: \"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"
Jul 17 18:03:14.180447 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:14.180424    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22\" (UID: \"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"
Jul 17 18:03:14.228643 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:14.228610    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"
Jul 17 18:03:14.373787 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:14.373757    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"]
Jul 17 18:03:14.376827 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:03:14.376798    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948e819d_fb3d_4f4e_83f5_bee8f2e9b8f4.slice/crio-2f6ec69a71261f8dfd9b24de1699274eec023acda9245a7bcda0c2a287705d22 WatchSource:0}: Error finding container 2f6ec69a71261f8dfd9b24de1699274eec023acda9245a7bcda0c2a287705d22: Status 404 returned error can't find the container with id 2f6ec69a71261f8dfd9b24de1699274eec023acda9245a7bcda0c2a287705d22
Jul 17 18:03:15.074629 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:15.074590    2572 generic.go:356] "Generic (PLEG): container finished" podID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerID="b7ac86faf0bca5dd87d01bee6d708d007342536fc8b81cb9fe95089f01e1b179" exitCode=7
Jul 17 18:03:15.075037 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:15.074681    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" event={"ID":"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4","Type":"ContainerDied","Data":"b7ac86faf0bca5dd87d01bee6d708d007342536fc8b81cb9fe95089f01e1b179"}
Jul 17 18:03:15.075037 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:15.074744    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" event={"ID":"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4","Type":"ContainerStarted","Data":"2f6ec69a71261f8dfd9b24de1699274eec023acda9245a7bcda0c2a287705d22"}
Jul 17 18:03:15.075037 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:15.074851    2572 scope.go:117] "RemoveContainer" containerID="b7ac86faf0bca5dd87d01bee6d708d007342536fc8b81cb9fe95089f01e1b179"
Jul 17 18:03:16.084937 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:16.084903    2572 generic.go:356] "Generic (PLEG): container finished" podID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerID="e4d64b5bab666bea8654177618b9078a8caaa00a098b5ea02a94a26a8447f460" exitCode=7
Jul 17 18:03:16.085434 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:16.084944    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" event={"ID":"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4","Type":"ContainerDied","Data":"e4d64b5bab666bea8654177618b9078a8caaa00a098b5ea02a94a26a8447f460"}
Jul 17 18:03:16.085434 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:16.084999    2572 scope.go:117] "RemoveContainer" containerID="b7ac86faf0bca5dd87d01bee6d708d007342536fc8b81cb9fe95089f01e1b179"
Jul 17 18:03:16.085434 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:16.085214    2572 scope.go:117] "RemoveContainer" containerID="e4d64b5bab666bea8654177618b9078a8caaa00a098b5ea02a94a26a8447f460"
Jul 17 18:03:16.085627 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:16.085482    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22_odh-ai-gateway-infra(948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4"
Jul 17 18:03:17.092132 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:17.092100    2572 scope.go:117] "RemoveContainer" containerID="e4d64b5bab666bea8654177618b9078a8caaa00a098b5ea02a94a26a8447f460"
Jul 17 18:03:17.092573 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:17.092322    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22_odh-ai-gateway-infra(948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4"
Jul 17 18:03:26.269081 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.269021    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"]
Jul 17 18:03:26.269725 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.269612    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerName="revoke-keys"
Jul 17 18:03:26.269725 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.269633    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="49aea314-c073-419f-b7c8-fd1834ed05b5" containerName="revoke-keys"
Jul 17 18:03:26.269725 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.269649    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerName="revoke-keys"
Jul 17 18:03:26.269725 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.269658    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerName="revoke-keys"
Jul 17 18:03:26.269956 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.269746    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e2ad2cc-5455-4c67-8de7-cd22d2b035e8" containerName="revoke-keys"
Jul 17 18:03:26.272740 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.272716    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"
Jul 17 18:03:26.281941 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.281850    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/203c4dce-5b2c-4275-afcd-5cecdf22ffa2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz\" (UID: \"203c4dce-5b2c-4275-afcd-5cecdf22ffa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"
Jul 17 18:03:26.283325 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.283298    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"]
Jul 17 18:03:26.382329 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.382264    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/203c4dce-5b2c-4275-afcd-5cecdf22ffa2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz\" (UID: \"203c4dce-5b2c-4275-afcd-5cecdf22ffa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"
Jul 17 18:03:26.382980 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.382958    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/203c4dce-5b2c-4275-afcd-5cecdf22ffa2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz\" (UID: \"203c4dce-5b2c-4275-afcd-5cecdf22ffa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"
Jul 17 18:03:26.583766 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.583732    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"
Jul 17 18:03:26.736267 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:26.736230    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"]
Jul 17 18:03:26.739093 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:03:26.739061    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203c4dce_5b2c_4275_afcd_5cecdf22ffa2.slice/crio-7294ab73d770176eefb40cd10e17c5a91f9e77428e8aafabdddacd9b92d567e7 WatchSource:0}: Error finding container 7294ab73d770176eefb40cd10e17c5a91f9e77428e8aafabdddacd9b92d567e7: Status 404 returned error can't find the container with id 7294ab73d770176eefb40cd10e17c5a91f9e77428e8aafabdddacd9b92d567e7
Jul 17 18:03:27.146687 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:27.146644    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" event={"ID":"203c4dce-5b2c-4275-afcd-5cecdf22ffa2","Type":"ContainerStarted","Data":"0925b9ee126b032586797921eb8347fbd6963a6deb843adb71b61f3a50586f7b"}
Jul 17 18:03:27.146857 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:27.146707    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" event={"ID":"203c4dce-5b2c-4275-afcd-5cecdf22ffa2","Type":"ContainerStarted","Data":"7294ab73d770176eefb40cd10e17c5a91f9e77428e8aafabdddacd9b92d567e7"}
Jul 17 18:03:27.184153 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:27.184086    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" podStartSLOduration=1.184068716 podStartE2EDuration="1.184068716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:27.182614387 +0000 UTC m=+2379.187970246" watchObservedRunningTime="2026-07-17 18:03:27.184068716 +0000 UTC m=+2379.189424615"
Jul 17 18:03:28.153369 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:28.153331    2572 generic.go:356] "Generic (PLEG): container finished" podID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerID="0925b9ee126b032586797921eb8347fbd6963a6deb843adb71b61f3a50586f7b" exitCode=6
Jul 17 18:03:28.153895 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:28.153427    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" event={"ID":"203c4dce-5b2c-4275-afcd-5cecdf22ffa2","Type":"ContainerDied","Data":"0925b9ee126b032586797921eb8347fbd6963a6deb843adb71b61f3a50586f7b"}
Jul 17 18:03:28.153895 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:28.153633    2572 scope.go:117] "RemoveContainer" containerID="0925b9ee126b032586797921eb8347fbd6963a6deb843adb71b61f3a50586f7b"
Jul 17 18:03:29.160407 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:29.160369    2572 generic.go:356] "Generic (PLEG): container finished" podID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerID="3848d1ce7ed675551769b3ef0750bd0f1c3ee8f3a32bb3eaea092f29e8364452" exitCode=6
Jul 17 18:03:29.160945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:29.160442    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" event={"ID":"203c4dce-5b2c-4275-afcd-5cecdf22ffa2","Type":"ContainerDied","Data":"3848d1ce7ed675551769b3ef0750bd0f1c3ee8f3a32bb3eaea092f29e8364452"}
Jul 17 18:03:29.160945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:29.160530    2572 scope.go:117] "RemoveContainer" containerID="0925b9ee126b032586797921eb8347fbd6963a6deb843adb71b61f3a50586f7b"
Jul 17 18:03:29.160945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:29.160741    2572 scope.go:117] "RemoveContainer" containerID="3848d1ce7ed675551769b3ef0750bd0f1c3ee8f3a32bb3eaea092f29e8364452"
Jul 17 18:03:29.161117 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:29.160989    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz_odh-ai-gateway-infra(203c4dce-5b2c-4275-afcd-5cecdf22ffa2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2"
Jul 17 18:03:30.167693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:30.167660    2572 scope.go:117] "RemoveContainer" containerID="3848d1ce7ed675551769b3ef0750bd0f1c3ee8f3a32bb3eaea092f29e8364452"
Jul 17 18:03:30.168131 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:30.167906    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz_odh-ai-gateway-infra(203c4dce-5b2c-4275-afcd-5cecdf22ffa2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2"
Jul 17 18:03:31.510455 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:31.510410    2572 scope.go:117] "RemoveContainer" containerID="e4d64b5bab666bea8654177618b9078a8caaa00a098b5ea02a94a26a8447f460"
Jul 17 18:03:32.181301 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:32.181202    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" event={"ID":"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4","Type":"ContainerStarted","Data":"a0ff1f0df247b0a42a0a7660e278b22ba23b57a9e738302be9d7b079c1f95c15"}
Jul 17 18:03:32.206080 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:32.206022    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" podStartSLOduration=19.206000963 podStartE2EDuration="19.206000963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:32.203482883 +0000 UTC m=+2384.208838738" watchObservedRunningTime="2026-07-17 18:03:32.206000963 +0000 UTC m=+2384.211356817"
Jul 17 18:03:33.188824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:33.188788    2572 generic.go:356] "Generic (PLEG): container finished" podID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerID="a0ff1f0df247b0a42a0a7660e278b22ba23b57a9e738302be9d7b079c1f95c15" exitCode=7
Jul 17 18:03:33.189308 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:33.188869    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" event={"ID":"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4","Type":"ContainerDied","Data":"a0ff1f0df247b0a42a0a7660e278b22ba23b57a9e738302be9d7b079c1f95c15"}
Jul 17 18:03:33.189308 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:33.188935    2572 scope.go:117] "RemoveContainer" containerID="e4d64b5bab666bea8654177618b9078a8caaa00a098b5ea02a94a26a8447f460"
Jul 17 18:03:33.189308 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:33.189218    2572 scope.go:117] "RemoveContainer" containerID="a0ff1f0df247b0a42a0a7660e278b22ba23b57a9e738302be9d7b079c1f95c15"
Jul 17 18:03:33.189455 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:33.189434    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22_odh-ai-gateway-infra(948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4"
Jul 17 18:03:33.221501 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:33.221465    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"]
Jul 17 18:03:34.340631 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:34.340598    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"
Jul 17 18:03:34.445678 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:34.445638    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4-maas-api-service-ca\") pod \"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4\" (UID: \"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4\") "
Jul 17 18:03:34.446064 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:34.446010    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" (UID: "948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:03:34.546250 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:34.546216    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:03:35.202366 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:35.202333    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"
Jul 17 18:03:35.202617 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:35.202378    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22" event={"ID":"948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4","Type":"ContainerDied","Data":"2f6ec69a71261f8dfd9b24de1699274eec023acda9245a7bcda0c2a287705d22"}
Jul 17 18:03:35.202617 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:35.202450    2572 scope.go:117] "RemoveContainer" containerID="a0ff1f0df247b0a42a0a7660e278b22ba23b57a9e738302be9d7b079c1f95c15"
Jul 17 18:03:35.223857 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:35.223820    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"]
Jul 17 18:03:35.231693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:35.231659    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-s5d22"]
Jul 17 18:03:36.515456 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:36.515420    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" path="/var/lib/kubelet/pods/948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4/volumes"
Jul 17 18:03:37.823433 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.823384    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"]
Jul 17 18:03:37.823894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.823813    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerName="revoke-keys"
Jul 17 18:03:37.823894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.823829    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerName="revoke-keys"
Jul 17 18:03:37.823894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.823848    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerName="revoke-keys"
Jul 17 18:03:37.823894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.823854    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerName="revoke-keys"
Jul 17 18:03:37.824028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.823920    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerName="revoke-keys"
Jul 17 18:03:37.824028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.823930    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerName="revoke-keys"
Jul 17 18:03:37.828274 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.828248    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"
Jul 17 18:03:37.842945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.842898    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"]
Jul 17 18:03:37.876274 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.876237    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a76c6e-e8e7-4a1c-99fd-2bdb43b81784-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx\" (UID: \"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"
Jul 17 18:03:37.976828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.976787    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a76c6e-e8e7-4a1c-99fd-2bdb43b81784-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx\" (UID: \"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"
Jul 17 18:03:37.977444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:37.977422    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a76c6e-e8e7-4a1c-99fd-2bdb43b81784-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx\" (UID: \"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"
Jul 17 18:03:38.139390 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:38.139297    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"
Jul 17 18:03:38.288801 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:38.288772    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"]
Jul 17 18:03:38.291367 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:03:38.291337    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a76c6e_e8e7_4a1c_99fd_2bdb43b81784.slice/crio-84c1cf9aff076cd556764574bcbccb5ce9d754d2e5d39b9ba130dbfd56065fb0 WatchSource:0}: Error finding container 84c1cf9aff076cd556764574bcbccb5ce9d754d2e5d39b9ba130dbfd56065fb0: Status 404 returned error can't find the container with id 84c1cf9aff076cd556764574bcbccb5ce9d754d2e5d39b9ba130dbfd56065fb0
Jul 17 18:03:39.227469 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:39.227433    2572 generic.go:356] "Generic (PLEG): container finished" podID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerID="5ce889d4d32bd484486da812d7b6347c74327fcdf28d3cfa2485daad92bfff92" exitCode=6
Jul 17 18:03:39.227928 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:39.227514    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx" event={"ID":"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784","Type":"ContainerDied","Data":"5ce889d4d32bd484486da812d7b6347c74327fcdf28d3cfa2485daad92bfff92"}
Jul 17 18:03:39.227928 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:39.227564    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx" event={"ID":"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784","Type":"ContainerStarted","Data":"84c1cf9aff076cd556764574bcbccb5ce9d754d2e5d39b9ba130dbfd56065fb0"}
Jul 17 18:03:39.227928 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:39.227781    2572 scope.go:117] "RemoveContainer" containerID="5ce889d4d32bd484486da812d7b6347c74327fcdf28d3cfa2485daad92bfff92"
Jul 17 18:03:40.234983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:40.234891    2572 generic.go:356] "Generic (PLEG): container finished" podID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerID="d7447d11d6ffe31b4f24dabb31d880188b5228a95060e8009622b4fae93e865b" exitCode=6
Jul 17 18:03:40.235400 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:40.234980    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx" event={"ID":"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784","Type":"ContainerDied","Data":"d7447d11d6ffe31b4f24dabb31d880188b5228a95060e8009622b4fae93e865b"}
Jul 17 18:03:40.235400 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:40.235048    2572 scope.go:117] "RemoveContainer" containerID="5ce889d4d32bd484486da812d7b6347c74327fcdf28d3cfa2485daad92bfff92"
Jul 17 18:03:40.235400 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:40.235305    2572 scope.go:117] "RemoveContainer" containerID="d7447d11d6ffe31b4f24dabb31d880188b5228a95060e8009622b4fae93e865b"
Jul 17 18:03:40.235573 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:40.235551    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx_odh-ai-gateway-infra(03a76c6e-e8e7-4a1c-99fd-2bdb43b81784)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784"
Jul 17 18:03:41.242918 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:41.242880    2572 scope.go:117] "RemoveContainer" containerID="d7447d11d6ffe31b4f24dabb31d880188b5228a95060e8009622b4fae93e865b"
Jul 17 18:03:41.243455 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:41.243140    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx_odh-ai-gateway-infra(03a76c6e-e8e7-4a1c-99fd-2bdb43b81784)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784"
Jul 17 18:03:41.510666 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:41.510561    2572 scope.go:117] "RemoveContainer" containerID="3848d1ce7ed675551769b3ef0750bd0f1c3ee8f3a32bb3eaea092f29e8364452"
Jul 17 18:03:42.249946 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:42.249907    2572 generic.go:356] "Generic (PLEG): container finished" podID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerID="3782c965040ee34b304065c9a18b2d38341c43791e03a75f9f0b1a1a66f5826d" exitCode=6
Jul 17 18:03:42.250402 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:42.249979    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" event={"ID":"203c4dce-5b2c-4275-afcd-5cecdf22ffa2","Type":"ContainerDied","Data":"3782c965040ee34b304065c9a18b2d38341c43791e03a75f9f0b1a1a66f5826d"}
Jul 17 18:03:42.250402 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:42.250042    2572 scope.go:117] "RemoveContainer" containerID="3848d1ce7ed675551769b3ef0750bd0f1c3ee8f3a32bb3eaea092f29e8364452"
Jul 17 18:03:42.250402 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:42.250358    2572 scope.go:117] "RemoveContainer" containerID="3782c965040ee34b304065c9a18b2d38341c43791e03a75f9f0b1a1a66f5826d"
Jul 17 18:03:42.250630 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:42.250610    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz_odh-ai-gateway-infra(203c4dce-5b2c-4275-afcd-5cecdf22ffa2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2"
Jul 17 18:03:43.309698 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:43.309655    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"]
Jul 17 18:03:43.452086 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:43.452061    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"
Jul 17 18:03:43.528426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:43.528388    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/203c4dce-5b2c-4275-afcd-5cecdf22ffa2-maas-api-service-ca\") pod \"203c4dce-5b2c-4275-afcd-5cecdf22ffa2\" (UID: \"203c4dce-5b2c-4275-afcd-5cecdf22ffa2\") "
Jul 17 18:03:43.528794 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:43.528771    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/203c4dce-5b2c-4275-afcd-5cecdf22ffa2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "203c4dce-5b2c-4275-afcd-5cecdf22ffa2" (UID: "203c4dce-5b2c-4275-afcd-5cecdf22ffa2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:03:43.629360 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:43.629250    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/203c4dce-5b2c-4275-afcd-5cecdf22ffa2-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:03:44.263895 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:44.263862    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"
Jul 17 18:03:44.264089 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:44.263892    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz" event={"ID":"203c4dce-5b2c-4275-afcd-5cecdf22ffa2","Type":"ContainerDied","Data":"7294ab73d770176eefb40cd10e17c5a91f9e77428e8aafabdddacd9b92d567e7"}
Jul 17 18:03:44.264089 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:44.263963    2572 scope.go:117] "RemoveContainer" containerID="3782c965040ee34b304065c9a18b2d38341c43791e03a75f9f0b1a1a66f5826d"
Jul 17 18:03:44.296881 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:44.296844    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"]
Jul 17 18:03:44.299235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:44.299204    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vf2dz"]
Jul 17 18:03:44.516537 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:44.516441    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" path="/var/lib/kubelet/pods/203c4dce-5b2c-4275-afcd-5cecdf22ffa2/volumes"
Jul 17 18:03:48.658862 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:48.658829    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 18:03:48.663931 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:48.663902    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 18:03:48.665412 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:48.665389    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 18:03:48.670583 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:48.670560    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 18:03:49.423544 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.423486    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"]
Jul 17 18:03:49.423893 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.423878    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerName="revoke-keys"
Jul 17 18:03:49.423944 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.423895    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerName="revoke-keys"
Jul 17 18:03:49.423944 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.423912    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerName="revoke-keys"
Jul 17 18:03:49.423944 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.423918    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerName="revoke-keys"
Jul 17 18:03:49.423944 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.423928    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerName="revoke-keys"
Jul 17 18:03:49.423944 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.423933    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerName="revoke-keys"
Jul 17 18:03:49.424096 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.424007    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerName="revoke-keys"
Jul 17 18:03:49.424096 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.424020    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="948e819d-fb3d-4f4e-83f5-bee8f2e9b8f4" containerName="revoke-keys"
Jul 17 18:03:49.424096 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.424026    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerName="revoke-keys"
Jul 17 18:03:49.428805 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.428783    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"
Jul 17 18:03:49.438570 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.438536    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"]
Jul 17 18:03:49.477909 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.477866    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4bd37b-491d-44a8-8668-3d26bb2c0866-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr\" (UID: \"4d4bd37b-491d-44a8-8668-3d26bb2c0866\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"
Jul 17 18:03:49.579396 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.579284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4bd37b-491d-44a8-8668-3d26bb2c0866-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr\" (UID: \"4d4bd37b-491d-44a8-8668-3d26bb2c0866\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"
Jul 17 18:03:49.579994 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.579965    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4bd37b-491d-44a8-8668-3d26bb2c0866-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr\" (UID: \"4d4bd37b-491d-44a8-8668-3d26bb2c0866\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"
Jul 17 18:03:49.741693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.741583    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"
Jul 17 18:03:49.893260 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:49.893233    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"]
Jul 17 18:03:49.895394 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:03:49.895349    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d4bd37b_491d_44a8_8668_3d26bb2c0866.slice/crio-fb164846665af4273d37b09859ddb264f6751e7f09646957a6ef6d815b054ed6 WatchSource:0}: Error finding container fb164846665af4273d37b09859ddb264f6751e7f09646957a6ef6d815b054ed6: Status 404 returned error can't find the container with id fb164846665af4273d37b09859ddb264f6751e7f09646957a6ef6d815b054ed6
Jul 17 18:03:50.299442 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:50.299410    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" event={"ID":"4d4bd37b-491d-44a8-8668-3d26bb2c0866","Type":"ContainerStarted","Data":"168af097ad3c5e299468b4280b7edb7eca5b51270270175d8ec39d0cd1f859e2"}
Jul 17 18:03:50.299646 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:50.299464    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" event={"ID":"4d4bd37b-491d-44a8-8668-3d26bb2c0866","Type":"ContainerStarted","Data":"fb164846665af4273d37b09859ddb264f6751e7f09646957a6ef6d815b054ed6"}
Jul 17 18:03:50.328685 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:50.328625    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" podStartSLOduration=1.328604257 podStartE2EDuration="1.328604257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:50.326814057 +0000 UTC m=+2402.332169913" watchObservedRunningTime="2026-07-17 18:03:50.328604257 +0000 UTC m=+2402.333960111"
Jul 17 18:03:51.306437 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:51.306392    2572 generic.go:356] "Generic (PLEG): container finished" podID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerID="168af097ad3c5e299468b4280b7edb7eca5b51270270175d8ec39d0cd1f859e2" exitCode=6
Jul 17 18:03:51.306875 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:51.306481    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" event={"ID":"4d4bd37b-491d-44a8-8668-3d26bb2c0866","Type":"ContainerDied","Data":"168af097ad3c5e299468b4280b7edb7eca5b51270270175d8ec39d0cd1f859e2"}
Jul 17 18:03:51.306875 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:51.306697    2572 scope.go:117] "RemoveContainer" containerID="168af097ad3c5e299468b4280b7edb7eca5b51270270175d8ec39d0cd1f859e2"
Jul 17 18:03:52.314244 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:52.314203    2572 generic.go:356] "Generic (PLEG): container finished" podID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerID="e6c77dfbbe17037c72d428e2dca52e4b13eb16ca52808091ee94e025704f3de5" exitCode=6
Jul 17 18:03:52.314726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:52.314302    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" event={"ID":"4d4bd37b-491d-44a8-8668-3d26bb2c0866","Type":"ContainerDied","Data":"e6c77dfbbe17037c72d428e2dca52e4b13eb16ca52808091ee94e025704f3de5"}
Jul 17 18:03:52.314726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:52.314424    2572 scope.go:117] "RemoveContainer" containerID="168af097ad3c5e299468b4280b7edb7eca5b51270270175d8ec39d0cd1f859e2"
Jul 17 18:03:52.314726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:52.314566    2572 scope.go:117] "RemoveContainer" containerID="e6c77dfbbe17037c72d428e2dca52e4b13eb16ca52808091ee94e025704f3de5"
Jul 17 18:03:52.314881 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:52.314813    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr_odh-ai-gateway-infra(4d4bd37b-491d-44a8-8668-3d26bb2c0866)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866"
Jul 17 18:03:52.509823 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:52.509777    2572 scope.go:117] "RemoveContainer" containerID="d7447d11d6ffe31b4f24dabb31d880188b5228a95060e8009622b4fae93e865b"
Jul 17 18:03:53.322288 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:53.322257    2572 scope.go:117] "RemoveContainer" containerID="e6c77dfbbe17037c72d428e2dca52e4b13eb16ca52808091ee94e025704f3de5"
Jul 17 18:03:53.322753 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:53.322551    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr_odh-ai-gateway-infra(4d4bd37b-491d-44a8-8668-3d26bb2c0866)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866"
Jul 17 18:03:53.323966 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:53.323941    2572 generic.go:356] "Generic (PLEG): container finished" podID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerID="588e75c991b08f87f412d70565526c6c50a84472b2a7d6d29e300510cbcbdc29" exitCode=6
Jul 17 18:03:53.324079 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:53.324010    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx" event={"ID":"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784","Type":"ContainerDied","Data":"588e75c991b08f87f412d70565526c6c50a84472b2a7d6d29e300510cbcbdc29"}
Jul 17 18:03:53.324079 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:53.324071    2572 scope.go:117] "RemoveContainer" containerID="d7447d11d6ffe31b4f24dabb31d880188b5228a95060e8009622b4fae93e865b"
Jul 17 18:03:53.324392 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:53.324374    2572 scope.go:117] "RemoveContainer" containerID="588e75c991b08f87f412d70565526c6c50a84472b2a7d6d29e300510cbcbdc29"
Jul 17 18:03:53.324619 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:03:53.324597    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx_odh-ai-gateway-infra(03a76c6e-e8e7-4a1c-99fd-2bdb43b81784)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784"
Jul 17 18:03:54.387785 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:54.387742    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"]
Jul 17 18:03:54.530700 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:54.530671    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"
Jul 17 18:03:54.626575 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:54.626539    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a76c6e-e8e7-4a1c-99fd-2bdb43b81784-maas-api-service-ca\") pod \"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784\" (UID: \"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784\") "
Jul 17 18:03:54.626926 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:54.626904    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03a76c6e-e8e7-4a1c-99fd-2bdb43b81784-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" (UID: "03a76c6e-e8e7-4a1c-99fd-2bdb43b81784"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:03:54.728088 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:54.727997    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a76c6e-e8e7-4a1c-99fd-2bdb43b81784-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:03:55.340938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:55.340898    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx" event={"ID":"03a76c6e-e8e7-4a1c-99fd-2bdb43b81784","Type":"ContainerDied","Data":"84c1cf9aff076cd556764574bcbccb5ce9d754d2e5d39b9ba130dbfd56065fb0"}
Jul 17 18:03:55.341126 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:55.340959    2572 scope.go:117] "RemoveContainer" containerID="588e75c991b08f87f412d70565526c6c50a84472b2a7d6d29e300510cbcbdc29"
Jul 17 18:03:55.341126 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:55.340913    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"
Jul 17 18:03:55.364472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:55.364441    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"]
Jul 17 18:03:55.371968 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:55.371932    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-m8nfx"]
Jul 17 18:03:56.515185 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:03:56.515141    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" path="/var/lib/kubelet/pods/03a76c6e-e8e7-4a1c-99fd-2bdb43b81784/volumes"
Jul 17 18:04:04.510289 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:04.510249    2572 scope.go:117] "RemoveContainer" containerID="e6c77dfbbe17037c72d428e2dca52e4b13eb16ca52808091ee94e025704f3de5"
Jul 17 18:04:05.395435 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:05.395397    2572 generic.go:356] "Generic (PLEG): container finished" podID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerID="a5be97411599f60f631f1a4df6507cb4e31fe894e62f1244c2096bffd85210e1" exitCode=6
Jul 17 18:04:05.395638 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:05.395474    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" event={"ID":"4d4bd37b-491d-44a8-8668-3d26bb2c0866","Type":"ContainerDied","Data":"a5be97411599f60f631f1a4df6507cb4e31fe894e62f1244c2096bffd85210e1"}
Jul 17 18:04:05.395638 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:05.395592    2572 scope.go:117] "RemoveContainer" containerID="e6c77dfbbe17037c72d428e2dca52e4b13eb16ca52808091ee94e025704f3de5"
Jul 17 18:04:05.395902 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:05.395887    2572 scope.go:117] "RemoveContainer" containerID="a5be97411599f60f631f1a4df6507cb4e31fe894e62f1244c2096bffd85210e1"
Jul 17 18:04:05.396155 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:05.396124    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr_odh-ai-gateway-infra(4d4bd37b-491d-44a8-8668-3d26bb2c0866)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866"
Jul 17 18:04:06.427977 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:06.427938    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"]
Jul 17 18:04:06.571794 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:06.571766    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"
Jul 17 18:04:06.639164 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:06.639119    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4bd37b-491d-44a8-8668-3d26bb2c0866-maas-api-service-ca\") pod \"4d4bd37b-491d-44a8-8668-3d26bb2c0866\" (UID: \"4d4bd37b-491d-44a8-8668-3d26bb2c0866\") "
Jul 17 18:04:06.639523 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:06.639465    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d4bd37b-491d-44a8-8668-3d26bb2c0866-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d4bd37b-491d-44a8-8668-3d26bb2c0866" (UID: "4d4bd37b-491d-44a8-8668-3d26bb2c0866"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:04:06.740189 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:06.740102    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4bd37b-491d-44a8-8668-3d26bb2c0866-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:04:07.410615 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:07.410577    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"
Jul 17 18:04:07.410848 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:07.410613    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr" event={"ID":"4d4bd37b-491d-44a8-8668-3d26bb2c0866","Type":"ContainerDied","Data":"fb164846665af4273d37b09859ddb264f6751e7f09646957a6ef6d815b054ed6"}
Jul 17 18:04:07.410848 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:07.410681    2572 scope.go:117] "RemoveContainer" containerID="a5be97411599f60f631f1a4df6507cb4e31fe894e62f1244c2096bffd85210e1"
Jul 17 18:04:07.438710 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:07.438672    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"]
Jul 17 18:04:07.444986 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:07.444956    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-gkqqr"]
Jul 17 18:04:08.515947 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:08.515913    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" path="/var/lib/kubelet/pods/4d4bd37b-491d-44a8-8668-3d26bb2c0866/volumes"
Jul 17 18:04:13.984404 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984312    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"]
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984705    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerName="revoke-keys"
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984720    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerName="revoke-keys"
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984734    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerName="revoke-keys"
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984744    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerName="revoke-keys"
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984758    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerName="revoke-keys"
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984765    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerName="revoke-keys"
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984775    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerName="revoke-keys"
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984781    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerName="revoke-keys"
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984788    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerName="revoke-keys"
Jul 17 18:04:13.984828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984794    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerName="revoke-keys"
Jul 17 18:04:13.985145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984860    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerName="revoke-keys"
Jul 17 18:04:13.985145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984868    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerName="revoke-keys"
Jul 17 18:04:13.985145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984874    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="203c4dce-5b2c-4275-afcd-5cecdf22ffa2" containerName="revoke-keys"
Jul 17 18:04:13.985145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984881    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerName="revoke-keys"
Jul 17 18:04:13.985145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984886    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerName="revoke-keys"
Jul 17 18:04:13.985145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984893    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerName="revoke-keys"
Jul 17 18:04:13.985145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.984903    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerName="revoke-keys"
Jul 17 18:04:13.989299 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.989277    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"
Jul 17 18:04:13.992613 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.992585    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:04:13.992862 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:13.992842    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:04:14.002666 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:14.002639    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"]
Jul 17 18:04:14.107433 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:14.107384    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c053c-d880-4070-97a3-9f7ad86d8150-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj\" (UID: \"b14c053c-d880-4070-97a3-9f7ad86d8150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"
Jul 17 18:04:14.208760 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:14.208709    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c053c-d880-4070-97a3-9f7ad86d8150-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj\" (UID: \"b14c053c-d880-4070-97a3-9f7ad86d8150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"
Jul 17 18:04:14.209420 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:14.209399    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c053c-d880-4070-97a3-9f7ad86d8150-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj\" (UID: \"b14c053c-d880-4070-97a3-9f7ad86d8150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"
Jul 17 18:04:14.300326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:14.300224    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"
Jul 17 18:04:14.452180 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:14.452141    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"]
Jul 17 18:04:14.455329 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:04:14.455301    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14c053c_d880_4070_97a3_9f7ad86d8150.slice/crio-bf399177072d329ed3d41b337d70761a166cfe846a9a4ca57dfc3bff33138fd2 WatchSource:0}: Error finding container bf399177072d329ed3d41b337d70761a166cfe846a9a4ca57dfc3bff33138fd2: Status 404 returned error can't find the container with id bf399177072d329ed3d41b337d70761a166cfe846a9a4ca57dfc3bff33138fd2
Jul 17 18:04:15.452616 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:15.452578    2572 generic.go:356] "Generic (PLEG): container finished" podID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerID="86b1d7d655f7c75c89e2f3d4e04025f64d94b82257fa6a64cd35e523f65e7875" exitCode=7
Jul 17 18:04:15.453045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:15.452655    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj" event={"ID":"b14c053c-d880-4070-97a3-9f7ad86d8150","Type":"ContainerDied","Data":"86b1d7d655f7c75c89e2f3d4e04025f64d94b82257fa6a64cd35e523f65e7875"}
Jul 17 18:04:15.453045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:15.452736    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj" event={"ID":"b14c053c-d880-4070-97a3-9f7ad86d8150","Type":"ContainerStarted","Data":"bf399177072d329ed3d41b337d70761a166cfe846a9a4ca57dfc3bff33138fd2"}
Jul 17 18:04:15.453045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:15.452977    2572 scope.go:117] "RemoveContainer" containerID="86b1d7d655f7c75c89e2f3d4e04025f64d94b82257fa6a64cd35e523f65e7875"
Jul 17 18:04:16.316203 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.316163    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"]
Jul 17 18:04:16.316547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.316533    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerName="revoke-keys"
Jul 17 18:04:16.316615 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.316548    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d4bd37b-491d-44a8-8668-3d26bb2c0866" containerName="revoke-keys"
Jul 17 18:04:16.316615 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.316567    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerName="revoke-keys"
Jul 17 18:04:16.316615 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.316573    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a76c6e-e8e7-4a1c-99fd-2bdb43b81784" containerName="revoke-keys"
Jul 17 18:04:16.319664 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.319645    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"
Jul 17 18:04:16.329133 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.329101    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"]
Jul 17 18:04:16.432357 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.432250    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b0a8f04-f98b-40db-b819-e11bd4a4d663-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-kghld\" (UID: \"9b0a8f04-f98b-40db-b819-e11bd4a4d663\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"
Jul 17 18:04:16.460043 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.460006    2572 generic.go:356] "Generic (PLEG): container finished" podID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerID="0706aa80a317605adadaecf56c916f409440bae64c82f987517a78d05d0f9335" exitCode=7
Jul 17 18:04:16.460585 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.460094    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj" event={"ID":"b14c053c-d880-4070-97a3-9f7ad86d8150","Type":"ContainerDied","Data":"0706aa80a317605adadaecf56c916f409440bae64c82f987517a78d05d0f9335"}
Jul 17 18:04:16.460585 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.460168    2572 scope.go:117] "RemoveContainer" containerID="86b1d7d655f7c75c89e2f3d4e04025f64d94b82257fa6a64cd35e523f65e7875"
Jul 17 18:04:16.460585 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.460413    2572 scope.go:117] "RemoveContainer" containerID="0706aa80a317605adadaecf56c916f409440bae64c82f987517a78d05d0f9335"
Jul 17 18:04:16.460778 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:16.460726    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj_odh-ai-gateway-infra(b14c053c-d880-4070-97a3-9f7ad86d8150)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150"
Jul 17 18:04:16.532883 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.532835    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b0a8f04-f98b-40db-b819-e11bd4a4d663-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-kghld\" (UID: \"9b0a8f04-f98b-40db-b819-e11bd4a4d663\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"
Jul 17 18:04:16.533517 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.533472    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b0a8f04-f98b-40db-b819-e11bd4a4d663-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-kghld\" (UID: \"9b0a8f04-f98b-40db-b819-e11bd4a4d663\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"
Jul 17 18:04:16.632047 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.631991    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"
Jul 17 18:04:16.775900 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:16.775873    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"]
Jul 17 18:04:16.777279 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:04:16.777249    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0a8f04_f98b_40db_b819_e11bd4a4d663.slice/crio-d1e1e5fab0f88f01cf4a0497edb2cb9d189be5bdabf4e00a7da36a3ae99b55f9 WatchSource:0}: Error finding container d1e1e5fab0f88f01cf4a0497edb2cb9d189be5bdabf4e00a7da36a3ae99b55f9: Status 404 returned error can't find the container with id d1e1e5fab0f88f01cf4a0497edb2cb9d189be5bdabf4e00a7da36a3ae99b55f9
Jul 17 18:04:17.466911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:17.466883    2572 scope.go:117] "RemoveContainer" containerID="0706aa80a317605adadaecf56c916f409440bae64c82f987517a78d05d0f9335"
Jul 17 18:04:17.467348 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:17.467147    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj_odh-ai-gateway-infra(b14c053c-d880-4070-97a3-9f7ad86d8150)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150"
Jul 17 18:04:17.468258 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:17.468231    2572 generic.go:356] "Generic (PLEG): container finished" podID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerID="daa910b31f01a2a646bba08edf31a056f61913fd94664e200daa7b9666d5b088" exitCode=6
Jul 17 18:04:17.468397 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:17.468319    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld" event={"ID":"9b0a8f04-f98b-40db-b819-e11bd4a4d663","Type":"ContainerDied","Data":"daa910b31f01a2a646bba08edf31a056f61913fd94664e200daa7b9666d5b088"}
Jul 17 18:04:17.468397 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:17.468382    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld" event={"ID":"9b0a8f04-f98b-40db-b819-e11bd4a4d663","Type":"ContainerStarted","Data":"d1e1e5fab0f88f01cf4a0497edb2cb9d189be5bdabf4e00a7da36a3ae99b55f9"}
Jul 17 18:04:17.468509 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:17.468481    2572 scope.go:117] "RemoveContainer" containerID="daa910b31f01a2a646bba08edf31a056f61913fd94664e200daa7b9666d5b088"
Jul 17 18:04:18.475818 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:18.475785    2572 generic.go:356] "Generic (PLEG): container finished" podID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerID="e4f14ac851b6cc2514ff0d1d2084592e2e6f2386fb3b2ee430337fa8a478331d" exitCode=6
Jul 17 18:04:18.476252 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:18.475867    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld" event={"ID":"9b0a8f04-f98b-40db-b819-e11bd4a4d663","Type":"ContainerDied","Data":"e4f14ac851b6cc2514ff0d1d2084592e2e6f2386fb3b2ee430337fa8a478331d"}
Jul 17 18:04:18.476252 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:18.475931    2572 scope.go:117] "RemoveContainer" containerID="daa910b31f01a2a646bba08edf31a056f61913fd94664e200daa7b9666d5b088"
Jul 17 18:04:18.476252 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:18.476139    2572 scope.go:117] "RemoveContainer" containerID="e4f14ac851b6cc2514ff0d1d2084592e2e6f2386fb3b2ee430337fa8a478331d"
Jul 17 18:04:18.476368 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:18.476331    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-kghld_odh-ai-gateway-infra(9b0a8f04-f98b-40db-b819-e11bd4a4d663)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663"
Jul 17 18:04:19.482434 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:19.482404    2572 scope.go:117] "RemoveContainer" containerID="e4f14ac851b6cc2514ff0d1d2084592e2e6f2386fb3b2ee430337fa8a478331d"
Jul 17 18:04:19.482869 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:19.482638    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-kghld_odh-ai-gateway-infra(9b0a8f04-f98b-40db-b819-e11bd4a4d663)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663"
Jul 17 18:04:27.884418 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:27.884381    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"]
Jul 17 18:04:27.887695 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:27.887677    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"
Jul 17 18:04:27.897712 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:27.897679    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"]
Jul 17 18:04:27.929392 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:27.929354    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-rg674\" (UID: \"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"
Jul 17 18:04:28.029862 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:28.029820    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-rg674\" (UID: \"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"
Jul 17 18:04:28.030515 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:28.030474    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-rg674\" (UID: \"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"
Jul 17 18:04:28.198305 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:28.198205    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"
Jul 17 18:04:28.347310 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:28.347277    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"]
Jul 17 18:04:28.349113 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:04:28.349085    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c6dc17_0ebe_48a8_a8aa_fbec3cecc4d5.slice/crio-d8a01a7392222fe224a8b07a18c58e86fe5aab7ed7bcd99fc92f622f096ca89b WatchSource:0}: Error finding container d8a01a7392222fe224a8b07a18c58e86fe5aab7ed7bcd99fc92f622f096ca89b: Status 404 returned error can't find the container with id d8a01a7392222fe224a8b07a18c58e86fe5aab7ed7bcd99fc92f622f096ca89b
Jul 17 18:04:28.535820 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:28.535712    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674" event={"ID":"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5","Type":"ContainerStarted","Data":"d8a01a7392222fe224a8b07a18c58e86fe5aab7ed7bcd99fc92f622f096ca89b"}
Jul 17 18:04:29.542893 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:29.542856    2572 generic.go:356] "Generic (PLEG): container finished" podID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerID="4cac78bb3a9a75cee185649bd5b3d8bebf8d68348098c218134ab47571508565" exitCode=6
Jul 17 18:04:29.543311 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:29.542941    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674" event={"ID":"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5","Type":"ContainerDied","Data":"4cac78bb3a9a75cee185649bd5b3d8bebf8d68348098c218134ab47571508565"}
Jul 17 18:04:29.543311 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:29.543093    2572 scope.go:117] "RemoveContainer" containerID="4cac78bb3a9a75cee185649bd5b3d8bebf8d68348098c218134ab47571508565"
Jul 17 18:04:30.549820 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:30.549724    2572 generic.go:356] "Generic (PLEG): container finished" podID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerID="129f6b5577edd0b4fe0bc38b682b70fa413e3b8e9f2e499e6a627a201f55fe71" exitCode=6
Jul 17 18:04:30.550315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:30.549817    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674" event={"ID":"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5","Type":"ContainerDied","Data":"129f6b5577edd0b4fe0bc38b682b70fa413e3b8e9f2e499e6a627a201f55fe71"}
Jul 17 18:04:30.550315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:30.549871    2572 scope.go:117] "RemoveContainer" containerID="4cac78bb3a9a75cee185649bd5b3d8bebf8d68348098c218134ab47571508565"
Jul 17 18:04:30.550315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:30.550002    2572 scope.go:117] "RemoveContainer" containerID="129f6b5577edd0b4fe0bc38b682b70fa413e3b8e9f2e499e6a627a201f55fe71"
Jul 17 18:04:30.550315 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:30.550268    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-rg674_odh-ai-gateway-infra(08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5"
Jul 17 18:04:31.557540 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:31.557466    2572 scope.go:117] "RemoveContainer" containerID="129f6b5577edd0b4fe0bc38b682b70fa413e3b8e9f2e499e6a627a201f55fe71"
Jul 17 18:04:31.557997 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:31.557745    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-rg674_odh-ai-gateway-infra(08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5"
Jul 17 18:04:32.510158 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:32.510119    2572 scope.go:117] "RemoveContainer" containerID="0706aa80a317605adadaecf56c916f409440bae64c82f987517a78d05d0f9335"
Jul 17 18:04:33.510101 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:33.510012    2572 scope.go:117] "RemoveContainer" containerID="e4f14ac851b6cc2514ff0d1d2084592e2e6f2386fb3b2ee430337fa8a478331d"
Jul 17 18:04:33.571618 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:33.571577    2572 generic.go:356] "Generic (PLEG): container finished" podID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerID="5624d552d5bdb014d5913c30ddee970fa474dd4b5b11da39347c27466a80f209" exitCode=7
Jul 17 18:04:33.571824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:33.571649    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj" event={"ID":"b14c053c-d880-4070-97a3-9f7ad86d8150","Type":"ContainerDied","Data":"5624d552d5bdb014d5913c30ddee970fa474dd4b5b11da39347c27466a80f209"}
Jul 17 18:04:33.571824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:33.571727    2572 scope.go:117] "RemoveContainer" containerID="0706aa80a317605adadaecf56c916f409440bae64c82f987517a78d05d0f9335"
Jul 17 18:04:33.572147 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:33.572120    2572 scope.go:117] "RemoveContainer" containerID="5624d552d5bdb014d5913c30ddee970fa474dd4b5b11da39347c27466a80f209"
Jul 17 18:04:33.572359 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:33.572341    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj_odh-ai-gateway-infra(b14c053c-d880-4070-97a3-9f7ad86d8150)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150"
Jul 17 18:04:34.579963 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:34.579929    2572 generic.go:356] "Generic (PLEG): container finished" podID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerID="72206feefae087e61b9a4b8cc779df0886089c12a2c1a9255229555e400cf37c" exitCode=6
Jul 17 18:04:34.580445 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:34.580009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld" event={"ID":"9b0a8f04-f98b-40db-b819-e11bd4a4d663","Type":"ContainerDied","Data":"72206feefae087e61b9a4b8cc779df0886089c12a2c1a9255229555e400cf37c"}
Jul 17 18:04:34.580445 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:34.580082    2572 scope.go:117] "RemoveContainer" containerID="e4f14ac851b6cc2514ff0d1d2084592e2e6f2386fb3b2ee430337fa8a478331d"
Jul 17 18:04:34.580445 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:34.580422    2572 scope.go:117] "RemoveContainer" containerID="72206feefae087e61b9a4b8cc779df0886089c12a2c1a9255229555e400cf37c"
Jul 17 18:04:34.580689 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:34.580667    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-kghld_odh-ai-gateway-infra(9b0a8f04-f98b-40db-b819-e11bd4a4d663)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663"
Jul 17 18:04:34.609886 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:34.609847    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"]
Jul 17 18:04:34.754980 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:34.754952    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"
Jul 17 18:04:34.785299 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:34.785266    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c053c-d880-4070-97a3-9f7ad86d8150-maas-api-service-ca\") pod \"b14c053c-d880-4070-97a3-9f7ad86d8150\" (UID: \"b14c053c-d880-4070-97a3-9f7ad86d8150\") "
Jul 17 18:04:34.785690 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:34.785664    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b14c053c-d880-4070-97a3-9f7ad86d8150-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b14c053c-d880-4070-97a3-9f7ad86d8150" (UID: "b14c053c-d880-4070-97a3-9f7ad86d8150"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:04:34.886923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:34.886832    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c053c-d880-4070-97a3-9f7ad86d8150-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:04:35.587024 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.586993    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"
Jul 17 18:04:35.587679 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.586996    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj" event={"ID":"b14c053c-d880-4070-97a3-9f7ad86d8150","Type":"ContainerDied","Data":"bf399177072d329ed3d41b337d70761a166cfe846a9a4ca57dfc3bff33138fd2"}
Jul 17 18:04:35.587679 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.587148    2572 scope.go:117] "RemoveContainer" containerID="5624d552d5bdb014d5913c30ddee970fa474dd4b5b11da39347c27466a80f209"
Jul 17 18:04:35.613814 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.613778    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"]
Jul 17 18:04:35.616168 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.616136    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"]
Jul 17 18:04:35.625951 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.625908    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bkfhj"]
Jul 17 18:04:35.784604 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.784575    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"
Jul 17 18:04:35.793937 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.793908    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b0a8f04-f98b-40db-b819-e11bd4a4d663-maas-api-service-ca\") pod \"9b0a8f04-f98b-40db-b819-e11bd4a4d663\" (UID: \"9b0a8f04-f98b-40db-b819-e11bd4a4d663\") "
Jul 17 18:04:35.794286 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.794262    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b0a8f04-f98b-40db-b819-e11bd4a4d663-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b0a8f04-f98b-40db-b819-e11bd4a4d663" (UID: "9b0a8f04-f98b-40db-b819-e11bd4a4d663"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:04:35.895010 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:35.894903    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b0a8f04-f98b-40db-b819-e11bd4a4d663-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:04:36.515372 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:36.515337    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" path="/var/lib/kubelet/pods/b14c053c-d880-4070-97a3-9f7ad86d8150/volumes"
Jul 17 18:04:36.596945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:36.596912    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"
Jul 17 18:04:36.596945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:36.596922    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld" event={"ID":"9b0a8f04-f98b-40db-b819-e11bd4a4d663","Type":"ContainerDied","Data":"d1e1e5fab0f88f01cf4a0497edb2cb9d189be5bdabf4e00a7da36a3ae99b55f9"}
Jul 17 18:04:36.597441 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:36.596987    2572 scope.go:117] "RemoveContainer" containerID="72206feefae087e61b9a4b8cc779df0886089c12a2c1a9255229555e400cf37c"
Jul 17 18:04:36.628744 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:36.628706    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"]
Jul 17 18:04:36.640711 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:36.640676    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-kghld"]
Jul 17 18:04:38.515965 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:38.515929    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" path="/var/lib/kubelet/pods/9b0a8f04-f98b-40db-b819-e11bd4a4d663/volumes"
Jul 17 18:04:39.488784 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.488748    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"]
Jul 17 18:04:39.489100 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489087    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerName="revoke-keys"
Jul 17 18:04:39.489162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489101    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerName="revoke-keys"
Jul 17 18:04:39.489162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489112    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerName="revoke-keys"
Jul 17 18:04:39.489162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489119    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerName="revoke-keys"
Jul 17 18:04:39.489162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489127    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerName="revoke-keys"
Jul 17 18:04:39.489162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489133    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerName="revoke-keys"
Jul 17 18:04:39.489162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489144    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerName="revoke-keys"
Jul 17 18:04:39.489162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489149    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerName="revoke-keys"
Jul 17 18:04:39.489162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489155    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerName="revoke-keys"
Jul 17 18:04:39.489162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489161    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerName="revoke-keys"
Jul 17 18:04:39.489470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489223    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerName="revoke-keys"
Jul 17 18:04:39.489470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489229    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerName="revoke-keys"
Jul 17 18:04:39.489470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489235    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerName="revoke-keys"
Jul 17 18:04:39.489470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.489244    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerName="revoke-keys"
Jul 17 18:04:39.493705 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.493684    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"
Jul 17 18:04:39.516683 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.516650    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"]
Jul 17 18:04:39.528032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.527996    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21dee4-c2ce-4b32-a6e7-e4d153214534-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4\" (UID: \"5f21dee4-c2ce-4b32-a6e7-e4d153214534\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"
Jul 17 18:04:39.628760 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.628721    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21dee4-c2ce-4b32-a6e7-e4d153214534-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4\" (UID: \"5f21dee4-c2ce-4b32-a6e7-e4d153214534\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"
Jul 17 18:04:39.629358 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.629338    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21dee4-c2ce-4b32-a6e7-e4d153214534-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4\" (UID: \"5f21dee4-c2ce-4b32-a6e7-e4d153214534\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"
Jul 17 18:04:39.804048 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.803938    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"
Jul 17 18:04:39.952926 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:39.952900    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"]
Jul 17 18:04:39.954670 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:04:39.954644    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f21dee4_c2ce_4b32_a6e7_e4d153214534.slice/crio-d23723a78d04ec8a80b7184004b8e15a1ab10aae18d87a56be1950ef011ec4ba WatchSource:0}: Error finding container d23723a78d04ec8a80b7184004b8e15a1ab10aae18d87a56be1950ef011ec4ba: Status 404 returned error can't find the container with id d23723a78d04ec8a80b7184004b8e15a1ab10aae18d87a56be1950ef011ec4ba
Jul 17 18:04:40.620457 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:40.620419    2572 generic.go:356] "Generic (PLEG): container finished" podID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerID="7e359969c89f61098c252dfae4d68f74c6c5e0a32fc8b673b01401c1cc21afe3" exitCode=6
Jul 17 18:04:40.620891 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:40.620529    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4" event={"ID":"5f21dee4-c2ce-4b32-a6e7-e4d153214534","Type":"ContainerDied","Data":"7e359969c89f61098c252dfae4d68f74c6c5e0a32fc8b673b01401c1cc21afe3"}
Jul 17 18:04:40.620891 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:40.620584    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4" event={"ID":"5f21dee4-c2ce-4b32-a6e7-e4d153214534","Type":"ContainerStarted","Data":"d23723a78d04ec8a80b7184004b8e15a1ab10aae18d87a56be1950ef011ec4ba"}
Jul 17 18:04:40.620891 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:40.620734    2572 scope.go:117] "RemoveContainer" containerID="7e359969c89f61098c252dfae4d68f74c6c5e0a32fc8b673b01401c1cc21afe3"
Jul 17 18:04:41.628393 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:41.628292    2572 generic.go:356] "Generic (PLEG): container finished" podID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerID="919d7acf6ffc1d9e0ca50f8900bda9957e64a4167b8682506a98baa722e30acc" exitCode=6
Jul 17 18:04:41.628393 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:41.628350    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4" event={"ID":"5f21dee4-c2ce-4b32-a6e7-e4d153214534","Type":"ContainerDied","Data":"919d7acf6ffc1d9e0ca50f8900bda9957e64a4167b8682506a98baa722e30acc"}
Jul 17 18:04:41.628895 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:41.628412    2572 scope.go:117] "RemoveContainer" containerID="7e359969c89f61098c252dfae4d68f74c6c5e0a32fc8b673b01401c1cc21afe3"
Jul 17 18:04:41.628895 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:41.628652    2572 scope.go:117] "RemoveContainer" containerID="919d7acf6ffc1d9e0ca50f8900bda9957e64a4167b8682506a98baa722e30acc"
Jul 17 18:04:41.628975 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:41.628894    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4_odh-ai-gateway-infra(5f21dee4-c2ce-4b32-a6e7-e4d153214534)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534"
Jul 17 18:04:42.635802 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:42.635778    2572 scope.go:117] "RemoveContainer" containerID="919d7acf6ffc1d9e0ca50f8900bda9957e64a4167b8682506a98baa722e30acc"
Jul 17 18:04:42.636206 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:42.635987    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4_odh-ai-gateway-infra(5f21dee4-c2ce-4b32-a6e7-e4d153214534)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534"
Jul 17 18:04:44.516168 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:44.516133    2572 scope.go:117] "RemoveContainer" containerID="129f6b5577edd0b4fe0bc38b682b70fa413e3b8e9f2e499e6a627a201f55fe71"
Jul 17 18:04:45.653927 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:45.653894    2572 generic.go:356] "Generic (PLEG): container finished" podID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerID="7fa6ec50374077234bdec3c4463d20124a7aa13a0ff7b6818a21aa0071242e74" exitCode=6
Jul 17 18:04:45.654346 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:45.653974    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674" event={"ID":"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5","Type":"ContainerDied","Data":"7fa6ec50374077234bdec3c4463d20124a7aa13a0ff7b6818a21aa0071242e74"}
Jul 17 18:04:45.654346 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:45.654065    2572 scope.go:117] "RemoveContainer" containerID="129f6b5577edd0b4fe0bc38b682b70fa413e3b8e9f2e499e6a627a201f55fe71"
Jul 17 18:04:45.654346 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:45.654328    2572 scope.go:117] "RemoveContainer" containerID="7fa6ec50374077234bdec3c4463d20124a7aa13a0ff7b6818a21aa0071242e74"
Jul 17 18:04:45.654610 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:45.654586    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-rg674_odh-ai-gateway-infra(08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5"
Jul 17 18:04:46.693320 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:46.693281    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"]
Jul 17 18:04:46.838164 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:46.838128    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"
Jul 17 18:04:46.892519 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:46.892456    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5-maas-api-service-ca\") pod \"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5\" (UID: \"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5\") "
Jul 17 18:04:46.892809 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:46.892785    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" (UID: "08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:04:46.993930 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:46.993844    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:04:47.670452 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:47.670413    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"
Jul 17 18:04:47.670688 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:47.670453    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674" event={"ID":"08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5","Type":"ContainerDied","Data":"d8a01a7392222fe224a8b07a18c58e86fe5aab7ed7bcd99fc92f622f096ca89b"}
Jul 17 18:04:47.670688 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:47.670538    2572 scope.go:117] "RemoveContainer" containerID="7fa6ec50374077234bdec3c4463d20124a7aa13a0ff7b6818a21aa0071242e74"
Jul 17 18:04:47.694865 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:47.694822    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"]
Jul 17 18:04:47.699589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:47.699544    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-rg674"]
Jul 17 18:04:48.515909 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:48.515871    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" path="/var/lib/kubelet/pods/08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5/volumes"
Jul 17 18:04:56.510682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:56.510648    2572 scope.go:117] "RemoveContainer" containerID="919d7acf6ffc1d9e0ca50f8900bda9957e64a4167b8682506a98baa722e30acc"
Jul 17 18:04:57.723739 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:57.723703    2572 generic.go:356] "Generic (PLEG): container finished" podID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerID="0937d124b661b05f0f168608414faa84e038d2ad85a5b60e2dcf643c06e8d855" exitCode=6
Jul 17 18:04:57.724255 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:57.723779    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4" event={"ID":"5f21dee4-c2ce-4b32-a6e7-e4d153214534","Type":"ContainerDied","Data":"0937d124b661b05f0f168608414faa84e038d2ad85a5b60e2dcf643c06e8d855"}
Jul 17 18:04:57.724255 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:57.723842    2572 scope.go:117] "RemoveContainer" containerID="919d7acf6ffc1d9e0ca50f8900bda9957e64a4167b8682506a98baa722e30acc"
Jul 17 18:04:57.724255 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:57.724226    2572 scope.go:117] "RemoveContainer" containerID="0937d124b661b05f0f168608414faa84e038d2ad85a5b60e2dcf643c06e8d855"
Jul 17 18:04:57.724472 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:04:57.724448    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4_odh-ai-gateway-infra(5f21dee4-c2ce-4b32-a6e7-e4d153214534)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534"
Jul 17 18:04:58.754461 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:58.754413    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"]
Jul 17 18:04:58.892622 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:58.892594    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"
Jul 17 18:04:59.009336 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:59.009228    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21dee4-c2ce-4b32-a6e7-e4d153214534-maas-api-service-ca\") pod \"5f21dee4-c2ce-4b32-a6e7-e4d153214534\" (UID: \"5f21dee4-c2ce-4b32-a6e7-e4d153214534\") "
Jul 17 18:04:59.009700 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:59.009676    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f21dee4-c2ce-4b32-a6e7-e4d153214534-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f21dee4-c2ce-4b32-a6e7-e4d153214534" (UID: "5f21dee4-c2ce-4b32-a6e7-e4d153214534"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:04:59.110010 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:59.109966    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f21dee4-c2ce-4b32-a6e7-e4d153214534-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:04:59.737258 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:59.737226    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"
Jul 17 18:04:59.737456 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:59.737267    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4" event={"ID":"5f21dee4-c2ce-4b32-a6e7-e4d153214534","Type":"ContainerDied","Data":"d23723a78d04ec8a80b7184004b8e15a1ab10aae18d87a56be1950ef011ec4ba"}
Jul 17 18:04:59.737456 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:59.737337    2572 scope.go:117] "RemoveContainer" containerID="0937d124b661b05f0f168608414faa84e038d2ad85a5b60e2dcf643c06e8d855"
Jul 17 18:04:59.760752 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:59.760721    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"]
Jul 17 18:04:59.765091 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:04:59.765056    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-j4jz4"]
Jul 17 18:05:00.516202 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:00.516154    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" path="/var/lib/kubelet/pods/5f21dee4-c2ce-4b32-a6e7-e4d153214534/volumes"
Jul 17 18:05:14.044593 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044554    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"]
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044882    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044893    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044902    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044907    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044920    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044926    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044935    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044940    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044955    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.044961    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.045009    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.045019    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.045025    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0a8f04-f98b-40db-b819-e11bd4a4d663" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.045031    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.045036    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.045040    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14c053c-d880-4070-97a3-9f7ad86d8150" containerName="revoke-keys"
Jul 17 18:05:14.045112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.045047    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerName="revoke-keys"
Jul 17 18:05:14.049478 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.049456    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"
Jul 17 18:05:14.054157 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.054129    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:05:14.054311 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.054188    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:05:14.078639 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.078600    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"]
Jul 17 18:05:14.145379 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.145335    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b6b72d-38d7-4996-b857-08c044cad5ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs\" (UID: \"d7b6b72d-38d7-4996-b857-08c044cad5ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"
Jul 17 18:05:14.246007 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.245973    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b6b72d-38d7-4996-b857-08c044cad5ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs\" (UID: \"d7b6b72d-38d7-4996-b857-08c044cad5ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"
Jul 17 18:05:14.246657 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.246638    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b6b72d-38d7-4996-b857-08c044cad5ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs\" (UID: \"d7b6b72d-38d7-4996-b857-08c044cad5ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"
Jul 17 18:05:14.360138 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.360097    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"
Jul 17 18:05:14.508485 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.508441    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"]
Jul 17 18:05:14.510001 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:05:14.509949    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b6b72d_38d7_4996_b857_08c044cad5ca.slice/crio-d7b19c02cd6e295e6317a93d47619202f4b56b73b92c06752e52da3e5ff0d19d WatchSource:0}: Error finding container d7b19c02cd6e295e6317a93d47619202f4b56b73b92c06752e52da3e5ff0d19d: Status 404 returned error can't find the container with id d7b19c02cd6e295e6317a93d47619202f4b56b73b92c06752e52da3e5ff0d19d
Jul 17 18:05:14.824055 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.824016    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" event={"ID":"d7b6b72d-38d7-4996-b857-08c044cad5ca","Type":"ContainerStarted","Data":"dfbd4283b5debe5fa5d5ba417b098e2950d6b07c6b33acf80232cafa8b4f66b4"}
Jul 17 18:05:14.824276 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.824076    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" event={"ID":"d7b6b72d-38d7-4996-b857-08c044cad5ca","Type":"ContainerStarted","Data":"d7b19c02cd6e295e6317a93d47619202f4b56b73b92c06752e52da3e5ff0d19d"}
Jul 17 18:05:14.842366 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:14.842304    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" podStartSLOduration=0.842287618 podStartE2EDuration="842.287618ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:05:14.841999923 +0000 UTC m=+2486.847355778" watchObservedRunningTime="2026-07-17 18:05:14.842287618 +0000 UTC m=+2486.847643473"
Jul 17 18:05:15.831027 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:15.830991    2572 generic.go:356] "Generic (PLEG): container finished" podID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerID="dfbd4283b5debe5fa5d5ba417b098e2950d6b07c6b33acf80232cafa8b4f66b4" exitCode=7
Jul 17 18:05:15.831027 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:15.831030    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" event={"ID":"d7b6b72d-38d7-4996-b857-08c044cad5ca","Type":"ContainerDied","Data":"dfbd4283b5debe5fa5d5ba417b098e2950d6b07c6b33acf80232cafa8b4f66b4"}
Jul 17 18:05:15.831515 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:15.831284    2572 scope.go:117] "RemoveContainer" containerID="dfbd4283b5debe5fa5d5ba417b098e2950d6b07c6b33acf80232cafa8b4f66b4"
Jul 17 18:05:16.367940 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.367902    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"]
Jul 17 18:05:16.368245 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.368233    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerName="revoke-keys"
Jul 17 18:05:16.368297 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.368246    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="08c6dc17-0ebe-48a8-a8aa-fbec3cecc4d5" containerName="revoke-keys"
Jul 17 18:05:16.368297 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.368262    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerName="revoke-keys"
Jul 17 18:05:16.368297 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.368268    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerName="revoke-keys"
Jul 17 18:05:16.368428 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.368324    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f21dee4-c2ce-4b32-a6e7-e4d153214534" containerName="revoke-keys"
Jul 17 18:05:16.371381 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.371362    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"
Jul 17 18:05:16.384511 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.384464    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"]
Jul 17 18:05:16.563884 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.563839    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9855b307-df4d-49e3-934a-df6edbe42476-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw\" (UID: \"9855b307-df4d-49e3-934a-df6edbe42476\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"
Jul 17 18:05:16.665381 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.665279    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9855b307-df4d-49e3-934a-df6edbe42476-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw\" (UID: \"9855b307-df4d-49e3-934a-df6edbe42476\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"
Jul 17 18:05:16.665971 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.665950    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9855b307-df4d-49e3-934a-df6edbe42476-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw\" (UID: \"9855b307-df4d-49e3-934a-df6edbe42476\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"
Jul 17 18:05:16.684336 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.684302    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"
Jul 17 18:05:16.839503 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.839461    2572 generic.go:356] "Generic (PLEG): container finished" podID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerID="cd73e27d237b158213dae1ea834ace2a768a65a5bacdefd1c864cd7111b1fc53" exitCode=7
Jul 17 18:05:16.839972 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.839533    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" event={"ID":"d7b6b72d-38d7-4996-b857-08c044cad5ca","Type":"ContainerDied","Data":"cd73e27d237b158213dae1ea834ace2a768a65a5bacdefd1c864cd7111b1fc53"}
Jul 17 18:05:16.839972 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.839596    2572 scope.go:117] "RemoveContainer" containerID="dfbd4283b5debe5fa5d5ba417b098e2950d6b07c6b33acf80232cafa8b4f66b4"
Jul 17 18:05:16.839972 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.839791    2572 scope.go:117] "RemoveContainer" containerID="cd73e27d237b158213dae1ea834ace2a768a65a5bacdefd1c864cd7111b1fc53"
Jul 17 18:05:16.840095 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:05:16.840036    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs_odh-ai-gateway-infra(d7b6b72d-38d7-4996-b857-08c044cad5ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca"
Jul 17 18:05:16.849376 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:16.849352    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"]
Jul 17 18:05:16.850856 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:05:16.850819    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9855b307_df4d_49e3_934a_df6edbe42476.slice/crio-228c166595a9d8fd9aeeb53c4310e246f503671d6008559d08f08b5012edd2a5 WatchSource:0}: Error finding container 228c166595a9d8fd9aeeb53c4310e246f503671d6008559d08f08b5012edd2a5: Status 404 returned error can't find the container with id 228c166595a9d8fd9aeeb53c4310e246f503671d6008559d08f08b5012edd2a5
Jul 17 18:05:17.845960 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:17.845923    2572 generic.go:356] "Generic (PLEG): container finished" podID="9855b307-df4d-49e3-934a-df6edbe42476" containerID="4668d7b9ad71699f06bf68a929ebee846f376f932b46e233074caf05efdce79d" exitCode=6
Jul 17 18:05:17.846442 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:17.846001    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw" event={"ID":"9855b307-df4d-49e3-934a-df6edbe42476","Type":"ContainerDied","Data":"4668d7b9ad71699f06bf68a929ebee846f376f932b46e233074caf05efdce79d"}
Jul 17 18:05:17.846442 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:17.846052    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw" event={"ID":"9855b307-df4d-49e3-934a-df6edbe42476","Type":"ContainerStarted","Data":"228c166595a9d8fd9aeeb53c4310e246f503671d6008559d08f08b5012edd2a5"}
Jul 17 18:05:17.846442 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:17.846199    2572 scope.go:117] "RemoveContainer" containerID="4668d7b9ad71699f06bf68a929ebee846f376f932b46e233074caf05efdce79d"
Jul 17 18:05:17.848004 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:17.847985    2572 scope.go:117] "RemoveContainer" containerID="cd73e27d237b158213dae1ea834ace2a768a65a5bacdefd1c864cd7111b1fc53"
Jul 17 18:05:17.848192 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:05:17.848170    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs_odh-ai-gateway-infra(d7b6b72d-38d7-4996-b857-08c044cad5ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca"
Jul 17 18:05:17.927897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:17.927859    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"]
Jul 17 18:05:17.932451 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:17.932432    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"
Jul 17 18:05:17.940398 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:17.940365    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"]
Jul 17 18:05:17.978900 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:17.978859    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5957fa63-b342-440e-abaf-33e62ebc15d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq\" (UID: \"5957fa63-b342-440e-abaf-33e62ebc15d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"
Jul 17 18:05:18.079964 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.079924    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5957fa63-b342-440e-abaf-33e62ebc15d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq\" (UID: \"5957fa63-b342-440e-abaf-33e62ebc15d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"
Jul 17 18:05:18.080605 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.080581    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5957fa63-b342-440e-abaf-33e62ebc15d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq\" (UID: \"5957fa63-b342-440e-abaf-33e62ebc15d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"
Jul 17 18:05:18.244830 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.244745    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"
Jul 17 18:05:18.387532 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.387503    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"]
Jul 17 18:05:18.389930 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:05:18.389893    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5957fa63_b342_440e_abaf_33e62ebc15d4.slice/crio-817ebe343617e7b42e680f2932f18d0b6aa00c362acf392d4cc21a1173a9dfcd WatchSource:0}: Error finding container 817ebe343617e7b42e680f2932f18d0b6aa00c362acf392d4cc21a1173a9dfcd: Status 404 returned error can't find the container with id 817ebe343617e7b42e680f2932f18d0b6aa00c362acf392d4cc21a1173a9dfcd
Jul 17 18:05:18.855436 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.855403    2572 generic.go:356] "Generic (PLEG): container finished" podID="9855b307-df4d-49e3-934a-df6edbe42476" containerID="14d637ad71e184293097b4505df857d2e04f1653fd01bac37516da6ee8828b02" exitCode=6
Jul 17 18:05:18.855983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.855512    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw" event={"ID":"9855b307-df4d-49e3-934a-df6edbe42476","Type":"ContainerDied","Data":"14d637ad71e184293097b4505df857d2e04f1653fd01bac37516da6ee8828b02"}
Jul 17 18:05:18.855983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.855589    2572 scope.go:117] "RemoveContainer" containerID="4668d7b9ad71699f06bf68a929ebee846f376f932b46e233074caf05efdce79d"
Jul 17 18:05:18.855983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.855824    2572 scope.go:117] "RemoveContainer" containerID="14d637ad71e184293097b4505df857d2e04f1653fd01bac37516da6ee8828b02"
Jul 17 18:05:18.856116 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:05:18.856077    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw_odh-ai-gateway-infra(9855b307-df4d-49e3-934a-df6edbe42476)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw" podUID="9855b307-df4d-49e3-934a-df6edbe42476"
Jul 17 18:05:18.857424 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.857397    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" event={"ID":"5957fa63-b342-440e-abaf-33e62ebc15d4","Type":"ContainerStarted","Data":"7ae2487e74921236411e39fbe890a14dfa893a4202301b9d9815b2fe76fda7cf"}
Jul 17 18:05:18.857568 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.857454    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" event={"ID":"5957fa63-b342-440e-abaf-33e62ebc15d4","Type":"ContainerStarted","Data":"817ebe343617e7b42e680f2932f18d0b6aa00c362acf392d4cc21a1173a9dfcd"}
Jul 17 18:05:18.898852 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:18.898789    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" podStartSLOduration=1.89876907 podStartE2EDuration="1.89876907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:05:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:05:18.897588289 +0000 UTC m=+2490.902944143" watchObservedRunningTime="2026-07-17 18:05:18.89876907 +0000 UTC m=+2490.904124992"
Jul 17 18:05:19.864035 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:19.863994    2572 generic.go:356] "Generic (PLEG): container finished" podID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerID="7ae2487e74921236411e39fbe890a14dfa893a4202301b9d9815b2fe76fda7cf" exitCode=6
Jul 17 18:05:19.864533 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:19.864081    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" event={"ID":"5957fa63-b342-440e-abaf-33e62ebc15d4","Type":"ContainerDied","Data":"7ae2487e74921236411e39fbe890a14dfa893a4202301b9d9815b2fe76fda7cf"}
Jul 17 18:05:19.864533 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:19.864305    2572 scope.go:117] "RemoveContainer" containerID="7ae2487e74921236411e39fbe890a14dfa893a4202301b9d9815b2fe76fda7cf"
Jul 17 18:05:19.866071 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:19.866055    2572 scope.go:117] "RemoveContainer" containerID="14d637ad71e184293097b4505df857d2e04f1653fd01bac37516da6ee8828b02"
Jul 17 18:05:19.866240 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:05:19.866220    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw_odh-ai-gateway-infra(9855b307-df4d-49e3-934a-df6edbe42476)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw" podUID="9855b307-df4d-49e3-934a-df6edbe42476"
Jul 17 18:05:20.874053 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:20.874010    2572 generic.go:356] "Generic (PLEG): container finished" podID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerID="ac46734f8688c041573e3d7edc5ea177f248d967d01c523e990294b84dcb11f7" exitCode=6
Jul 17 18:05:20.874572 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:20.874089    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" event={"ID":"5957fa63-b342-440e-abaf-33e62ebc15d4","Type":"ContainerDied","Data":"ac46734f8688c041573e3d7edc5ea177f248d967d01c523e990294b84dcb11f7"}
Jul 17 18:05:20.874572 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:20.874200    2572 scope.go:117] "RemoveContainer" containerID="7ae2487e74921236411e39fbe890a14dfa893a4202301b9d9815b2fe76fda7cf"
Jul 17 18:05:20.874572 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:20.874468    2572 scope.go:117] "RemoveContainer" containerID="ac46734f8688c041573e3d7edc5ea177f248d967d01c523e990294b84dcb11f7"
Jul 17 18:05:20.874761 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:05:20.874695    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq_odh-ai-gateway-infra(5957fa63-b342-440e-abaf-33e62ebc15d4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4"
Jul 17 18:05:21.881979 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:21.881952    2572 scope.go:117] "RemoveContainer" containerID="ac46734f8688c041573e3d7edc5ea177f248d967d01c523e990294b84dcb11f7"
Jul 17 18:05:21.882388 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:05:21.882163    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq_odh-ai-gateway-infra(5957fa63-b342-440e-abaf-33e62ebc15d4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4"
Jul 17 18:05:30.509845 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:30.509801    2572 scope.go:117] "RemoveContainer" containerID="cd73e27d237b158213dae1ea834ace2a768a65a5bacdefd1c864cd7111b1fc53"
Jul 17 18:05:30.932766 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:30.932732    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" event={"ID":"d7b6b72d-38d7-4996-b857-08c044cad5ca","Type":"ContainerStarted","Data":"ef2f992733e58e046ef2e06eff35c789bd4b2327d9b123ad48866bde9843aad5"}
Jul 17 18:05:31.940713 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:31.940671    2572 generic.go:356] "Generic (PLEG): container finished" podID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerID="ef2f992733e58e046ef2e06eff35c789bd4b2327d9b123ad48866bde9843aad5" exitCode=6
Jul 17 18:05:31.941215 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:31.940740    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" event={"ID":"d7b6b72d-38d7-4996-b857-08c044cad5ca","Type":"ContainerDied","Data":"ef2f992733e58e046ef2e06eff35c789bd4b2327d9b123ad48866bde9843aad5"}
Jul 17 18:05:31.941215 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:31.940804    2572 scope.go:117] "RemoveContainer" containerID="cd73e27d237b158213dae1ea834ace2a768a65a5bacdefd1c864cd7111b1fc53"
Jul 17 18:05:31.941215 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:31.941087    2572 scope.go:117] "RemoveContainer" containerID="ef2f992733e58e046ef2e06eff35c789bd4b2327d9b123ad48866bde9843aad5"
Jul 17 18:05:31.941448 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:05:31.941299    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs_odh-ai-gateway-infra(d7b6b72d-38d7-4996-b857-08c044cad5ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca"
Jul 17 18:05:31.963638 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:31.963602    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"]
Jul 17 18:05:33.084264 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:33.084230    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"
Jul 17 18:05:33.086076 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:33.086053    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b6b72d-38d7-4996-b857-08c044cad5ca-maas-api-service-ca\") pod \"d7b6b72d-38d7-4996-b857-08c044cad5ca\" (UID: \"d7b6b72d-38d7-4996-b857-08c044cad5ca\") "
Jul 17 18:05:33.086420 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:33.086399    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7b6b72d-38d7-4996-b857-08c044cad5ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d7b6b72d-38d7-4996-b857-08c044cad5ca" (UID: "d7b6b72d-38d7-4996-b857-08c044cad5ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:05:33.186681 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:33.186644    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7b6b72d-38d7-4996-b857-08c044cad5ca-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:05:33.956591 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:33.956552    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"
Jul 17 18:05:33.956759 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:33.956550    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs" event={"ID":"d7b6b72d-38d7-4996-b857-08c044cad5ca","Type":"ContainerDied","Data":"d7b19c02cd6e295e6317a93d47619202f4b56b73b92c06752e52da3e5ff0d19d"}
Jul 17 18:05:33.956759 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:33.956715    2572 scope.go:117] "RemoveContainer" containerID="ef2f992733e58e046ef2e06eff35c789bd4b2327d9b123ad48866bde9843aad5"
Jul 17 18:05:33.989202 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:33.989162    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"]
Jul 17 18:05:33.998649 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:33.998615    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-xpgvs"]
Jul 17 18:05:34.510068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:34.510039    2572 scope.go:117] "RemoveContainer" containerID="14d637ad71e184293097b4505df857d2e04f1653fd01bac37516da6ee8828b02"
Jul 17 18:05:34.515923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:34.515893    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" path="/var/lib/kubelet/pods/d7b6b72d-38d7-4996-b857-08c044cad5ca/volumes"
Jul 17 18:05:34.963980 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:34.963945    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw" event={"ID":"9855b307-df4d-49e3-934a-df6edbe42476","Type":"ContainerStarted","Data":"e6522ea1473a34d501c35d95dffc160e06258eeab7aeb67f74bef2ba4aa4680a"}
Jul 17 18:05:35.972449 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:35.972413    2572 generic.go:356] "Generic (PLEG): container finished" podID="9855b307-df4d-49e3-934a-df6edbe42476" containerID="e6522ea1473a34d501c35d95dffc160e06258eeab7aeb67f74bef2ba4aa4680a" exitCode=6
Jul 17 18:05:35.972893 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:35.972479    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw" event={"ID":"9855b307-df4d-49e3-934a-df6edbe42476","Type":"ContainerDied","Data":"e6522ea1473a34d501c35d95dffc160e06258eeab7aeb67f74bef2ba4aa4680a"}
Jul 17 18:05:35.972893 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:35.972567    2572 scope.go:117] "RemoveContainer" containerID="14d637ad71e184293097b4505df857d2e04f1653fd01bac37516da6ee8828b02"
Jul 17 18:05:35.972893 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:35.972856    2572 scope.go:117] "RemoveContainer" containerID="e6522ea1473a34d501c35d95dffc160e06258eeab7aeb67f74bef2ba4aa4680a"
Jul 17 18:05:35.973086 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:05:35.973065    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw_odh-ai-gateway-infra(9855b307-df4d-49e3-934a-df6edbe42476)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw" podUID="9855b307-df4d-49e3-934a-df6edbe42476"
Jul 17 18:05:36.005465 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:36.005428    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"]
Jul 17 18:05:36.510153 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:36.510111    2572 scope.go:117] "RemoveContainer" containerID="ac46734f8688c041573e3d7edc5ea177f248d967d01c523e990294b84dcb11f7"
Jul 17 18:05:36.979571 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:36.979527    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" event={"ID":"5957fa63-b342-440e-abaf-33e62ebc15d4","Type":"ContainerStarted","Data":"59627c557c85361b91dea208adcc5e0a996763bf070175243c5697b9e7fe0406"}
Jul 17 18:05:37.126384 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.126354    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"
Jul 17 18:05:37.217665 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.217624    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9855b307-df4d-49e3-934a-df6edbe42476-maas-api-service-ca\") pod \"9855b307-df4d-49e3-934a-df6edbe42476\" (UID: \"9855b307-df4d-49e3-934a-df6edbe42476\") "
Jul 17 18:05:37.218002 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.217979    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9855b307-df4d-49e3-934a-df6edbe42476-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9855b307-df4d-49e3-934a-df6edbe42476" (UID: "9855b307-df4d-49e3-934a-df6edbe42476"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:05:37.318860 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.318820    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9855b307-df4d-49e3-934a-df6edbe42476-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:05:37.987911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.987874    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"
Jul 17 18:05:37.988435 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.987923    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw" event={"ID":"9855b307-df4d-49e3-934a-df6edbe42476","Type":"ContainerDied","Data":"228c166595a9d8fd9aeeb53c4310e246f503671d6008559d08f08b5012edd2a5"}
Jul 17 18:05:37.988435 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.987997    2572 scope.go:117] "RemoveContainer" containerID="e6522ea1473a34d501c35d95dffc160e06258eeab7aeb67f74bef2ba4aa4680a"
Jul 17 18:05:37.990072 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.990046    2572 generic.go:356] "Generic (PLEG): container finished" podID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerID="59627c557c85361b91dea208adcc5e0a996763bf070175243c5697b9e7fe0406" exitCode=6
Jul 17 18:05:37.990193 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.990106    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" event={"ID":"5957fa63-b342-440e-abaf-33e62ebc15d4","Type":"ContainerDied","Data":"59627c557c85361b91dea208adcc5e0a996763bf070175243c5697b9e7fe0406"}
Jul 17 18:05:37.990411 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:37.990393    2572 scope.go:117] "RemoveContainer" containerID="59627c557c85361b91dea208adcc5e0a996763bf070175243c5697b9e7fe0406"
Jul 17 18:05:37.990670 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:05:37.990649    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq_odh-ai-gateway-infra(5957fa63-b342-440e-abaf-33e62ebc15d4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4"
Jul 17 18:05:38.003657 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:38.003634    2572 scope.go:117] "RemoveContainer" containerID="ac46734f8688c041573e3d7edc5ea177f248d967d01c523e990294b84dcb11f7"
Jul 17 18:05:38.036146 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:38.036004    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"]
Jul 17 18:05:38.038193 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:38.038171    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"]
Jul 17 18:05:38.042925 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:38.042890    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hw7nw"]
Jul 17 18:05:38.516021 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:38.515987    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9855b307-df4d-49e3-934a-df6edbe42476" path="/var/lib/kubelet/pods/9855b307-df4d-49e3-934a-df6edbe42476/volumes"
Jul 17 18:05:39.140775 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:39.140746    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"
Jul 17 18:05:39.233357 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:39.233319    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5957fa63-b342-440e-abaf-33e62ebc15d4-maas-api-service-ca\") pod \"5957fa63-b342-440e-abaf-33e62ebc15d4\" (UID: \"5957fa63-b342-440e-abaf-33e62ebc15d4\") "
Jul 17 18:05:39.233743 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:39.233718    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5957fa63-b342-440e-abaf-33e62ebc15d4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5957fa63-b342-440e-abaf-33e62ebc15d4" (UID: "5957fa63-b342-440e-abaf-33e62ebc15d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:05:39.333889 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:39.333842    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5957fa63-b342-440e-abaf-33e62ebc15d4-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:05:40.004690 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:40.004656    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"
Jul 17 18:05:40.004690 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:40.004680    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq" event={"ID":"5957fa63-b342-440e-abaf-33e62ebc15d4","Type":"ContainerDied","Data":"817ebe343617e7b42e680f2932f18d0b6aa00c362acf392d4cc21a1173a9dfcd"}
Jul 17 18:05:40.004924 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:40.004737    2572 scope.go:117] "RemoveContainer" containerID="59627c557c85361b91dea208adcc5e0a996763bf070175243c5697b9e7fe0406"
Jul 17 18:05:40.034111 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:40.034072    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"]
Jul 17 18:05:40.039999 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:40.039968    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-kkvhq"]
Jul 17 18:05:40.515697 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:05:40.515594    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" path="/var/lib/kubelet/pods/5957fa63-b342-440e-abaf-33e62ebc15d4/volumes"
Jul 17 18:06:17.992087 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992045    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"]
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992397    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992408    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992416    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992422    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992432    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9855b307-df4d-49e3-934a-df6edbe42476" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992441    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9855b307-df4d-49e3-934a-df6edbe42476" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992447    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992452    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992459    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9855b307-df4d-49e3-934a-df6edbe42476" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992464    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9855b307-df4d-49e3-934a-df6edbe42476" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992471    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9855b307-df4d-49e3-934a-df6edbe42476" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992475    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9855b307-df4d-49e3-934a-df6edbe42476" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992485    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992505    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992514    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992521    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992585    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992593    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9855b307-df4d-49e3-934a-df6edbe42476" containerName="revoke-keys"
Jul 17 18:06:17.992589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992600    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9855b307-df4d-49e3-934a-df6edbe42476" containerName="revoke-keys"
Jul 17 18:06:17.993601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992608    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerName="revoke-keys"
Jul 17 18:06:17.993601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992615    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerName="revoke-keys"
Jul 17 18:06:17.993601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992621    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerName="revoke-keys"
Jul 17 18:06:17.993601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.992627    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerName="revoke-keys"
Jul 17 18:06:17.995721 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.995703    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"
Jul 17 18:06:17.998832 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.998810    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:06:17.999061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:17.999045    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:06:18.004458 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:18.004434    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"]
Jul 17 18:06:18.076161 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:18.076122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b62292b-89c5-44a7-9028-619ac8c0f6a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9\" (UID: \"3b62292b-89c5-44a7-9028-619ac8c0f6a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"
Jul 17 18:06:18.177462 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:18.177415    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b62292b-89c5-44a7-9028-619ac8c0f6a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9\" (UID: \"3b62292b-89c5-44a7-9028-619ac8c0f6a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"
Jul 17 18:06:18.178172 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:18.178149    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b62292b-89c5-44a7-9028-619ac8c0f6a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9\" (UID: \"3b62292b-89c5-44a7-9028-619ac8c0f6a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"
Jul 17 18:06:18.308045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:18.308009    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"
Jul 17 18:06:18.467630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:18.467553    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"]
Jul 17 18:06:18.470077 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:06:18.470045    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b62292b_89c5_44a7_9028_619ac8c0f6a0.slice/crio-4614008165adf1f4a88ce9caa3f7b57ef7ebddaaf49849af3a0469d9f46867a4 WatchSource:0}: Error finding container 4614008165adf1f4a88ce9caa3f7b57ef7ebddaaf49849af3a0469d9f46867a4: Status 404 returned error can't find the container with id 4614008165adf1f4a88ce9caa3f7b57ef7ebddaaf49849af3a0469d9f46867a4
Jul 17 18:06:19.221560 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:19.221510    2572 generic.go:356] "Generic (PLEG): container finished" podID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerID="1574ae4d1cd8252a24db05ef2f1232a8d1349d0555211665dc38ed1222b1a122" exitCode=6
Jul 17 18:06:19.222010 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:19.221568    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9" event={"ID":"3b62292b-89c5-44a7-9028-619ac8c0f6a0","Type":"ContainerDied","Data":"1574ae4d1cd8252a24db05ef2f1232a8d1349d0555211665dc38ed1222b1a122"}
Jul 17 18:06:19.222010 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:19.221673    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9" event={"ID":"3b62292b-89c5-44a7-9028-619ac8c0f6a0","Type":"ContainerStarted","Data":"4614008165adf1f4a88ce9caa3f7b57ef7ebddaaf49849af3a0469d9f46867a4"}
Jul 17 18:06:19.222010 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:19.221900    2572 scope.go:117] "RemoveContainer" containerID="1574ae4d1cd8252a24db05ef2f1232a8d1349d0555211665dc38ed1222b1a122"
Jul 17 18:06:20.228347 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:20.228247    2572 generic.go:356] "Generic (PLEG): container finished" podID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerID="596ad48c4f5a18e86c3157f6c7c993adff36e433b3277067365597a75946c2b6" exitCode=6
Jul 17 18:06:20.228347 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:20.228287    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9" event={"ID":"3b62292b-89c5-44a7-9028-619ac8c0f6a0","Type":"ContainerDied","Data":"596ad48c4f5a18e86c3157f6c7c993adff36e433b3277067365597a75946c2b6"}
Jul 17 18:06:20.228871 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:20.228359    2572 scope.go:117] "RemoveContainer" containerID="1574ae4d1cd8252a24db05ef2f1232a8d1349d0555211665dc38ed1222b1a122"
Jul 17 18:06:20.228871 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:20.228541    2572 scope.go:117] "RemoveContainer" containerID="596ad48c4f5a18e86c3157f6c7c993adff36e433b3277067365597a75946c2b6"
Jul 17 18:06:20.228871 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:06:20.228749    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9_odh-ai-gateway-infra(3b62292b-89c5-44a7-9028-619ac8c0f6a0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0"
Jul 17 18:06:21.235027 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:21.234995    2572 scope.go:117] "RemoveContainer" containerID="596ad48c4f5a18e86c3157f6c7c993adff36e433b3277067365597a75946c2b6"
Jul 17 18:06:21.235444 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:06:21.235180    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9_odh-ai-gateway-infra(3b62292b-89c5-44a7-9028-619ac8c0f6a0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0"
Jul 17 18:06:34.509639 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:34.509605    2572 scope.go:117] "RemoveContainer" containerID="596ad48c4f5a18e86c3157f6c7c993adff36e433b3277067365597a75946c2b6"
Jul 17 18:06:35.310772 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:35.310733    2572 generic.go:356] "Generic (PLEG): container finished" podID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerID="7ec7423e05a791604b35f390a81a3a182cf8073fc3ebfc271dcd68e1a3c70231" exitCode=6
Jul 17 18:06:35.310969 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:35.310807    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9" event={"ID":"3b62292b-89c5-44a7-9028-619ac8c0f6a0","Type":"ContainerDied","Data":"7ec7423e05a791604b35f390a81a3a182cf8073fc3ebfc271dcd68e1a3c70231"}
Jul 17 18:06:35.310969 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:35.310870    2572 scope.go:117] "RemoveContainer" containerID="596ad48c4f5a18e86c3157f6c7c993adff36e433b3277067365597a75946c2b6"
Jul 17 18:06:35.311207 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:35.311184    2572 scope.go:117] "RemoveContainer" containerID="7ec7423e05a791604b35f390a81a3a182cf8073fc3ebfc271dcd68e1a3c70231"
Jul 17 18:06:35.311479 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:06:35.311457    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9_odh-ai-gateway-infra(3b62292b-89c5-44a7-9028-619ac8c0f6a0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0"
Jul 17 18:06:36.346840 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:36.346802    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"]
Jul 17 18:06:36.490944 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:36.490914    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"
Jul 17 18:06:36.638653 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:36.638561    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b62292b-89c5-44a7-9028-619ac8c0f6a0-maas-api-service-ca\") pod \"3b62292b-89c5-44a7-9028-619ac8c0f6a0\" (UID: \"3b62292b-89c5-44a7-9028-619ac8c0f6a0\") "
Jul 17 18:06:36.638921 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:36.638898    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b62292b-89c5-44a7-9028-619ac8c0f6a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b62292b-89c5-44a7-9028-619ac8c0f6a0" (UID: "3b62292b-89c5-44a7-9028-619ac8c0f6a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:06:36.739440 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:36.739399    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b62292b-89c5-44a7-9028-619ac8c0f6a0-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:06:37.324284 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:37.324257    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"
Jul 17 18:06:37.324514 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:37.324260    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9" event={"ID":"3b62292b-89c5-44a7-9028-619ac8c0f6a0","Type":"ContainerDied","Data":"4614008165adf1f4a88ce9caa3f7b57ef7ebddaaf49849af3a0469d9f46867a4"}
Jul 17 18:06:37.324514 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:37.324385    2572 scope.go:117] "RemoveContainer" containerID="7ec7423e05a791604b35f390a81a3a182cf8073fc3ebfc271dcd68e1a3c70231"
Jul 17 18:06:37.352923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:37.352877    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"]
Jul 17 18:06:37.353614 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:37.353587    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6rd9"]
Jul 17 18:06:38.515668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:38.515635    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" path="/var/lib/kubelet/pods/3b62292b-89c5-44a7-9028-619ac8c0f6a0/volumes"
Jul 17 18:06:39.599910 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.599868    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"]
Jul 17 18:06:39.600419 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600330    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerName="revoke-keys"
Jul 17 18:06:39.600419 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600349    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerName="revoke-keys"
Jul 17 18:06:39.600419 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600362    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerName="revoke-keys"
Jul 17 18:06:39.600419 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600369    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerName="revoke-keys"
Jul 17 18:06:39.600419 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600391    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerName="revoke-keys"
Jul 17 18:06:39.600419 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600400    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5957fa63-b342-440e-abaf-33e62ebc15d4" containerName="revoke-keys"
Jul 17 18:06:39.600781 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600512    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9855b307-df4d-49e3-934a-df6edbe42476" containerName="revoke-keys"
Jul 17 18:06:39.600781 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600532    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerName="revoke-keys"
Jul 17 18:06:39.600781 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600541    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerName="revoke-keys"
Jul 17 18:06:39.600781 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.600549    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7b6b72d-38d7-4996-b857-08c044cad5ca" containerName="revoke-keys"
Jul 17 18:06:39.605078 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.605052    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"
Jul 17 18:06:39.608139 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.608117    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:06:39.608265 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.608160    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:06:39.615172 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.615144    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"]
Jul 17 18:06:39.657871 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.657836    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/747c5890-78e9-41bf-b864-5496e87ec403-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-7wftf\" (UID: \"747c5890-78e9-41bf-b864-5496e87ec403\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"
Jul 17 18:06:39.759365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.759330    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/747c5890-78e9-41bf-b864-5496e87ec403-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-7wftf\" (UID: \"747c5890-78e9-41bf-b864-5496e87ec403\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"
Jul 17 18:06:39.760016 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.759995    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/747c5890-78e9-41bf-b864-5496e87ec403-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-7wftf\" (UID: \"747c5890-78e9-41bf-b864-5496e87ec403\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"
Jul 17 18:06:39.917012 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:39.916906    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"
Jul 17 18:06:40.058675 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:40.058641    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"]
Jul 17 18:06:40.060472 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:06:40.060447    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod747c5890_78e9_41bf_b864_5496e87ec403.slice/crio-67dbe3238ee26a24a06e998eedbb35dd8e6d46e214ca9951ccf88da914352e09 WatchSource:0}: Error finding container 67dbe3238ee26a24a06e998eedbb35dd8e6d46e214ca9951ccf88da914352e09: Status 404 returned error can't find the container with id 67dbe3238ee26a24a06e998eedbb35dd8e6d46e214ca9951ccf88da914352e09
Jul 17 18:06:40.341705 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:40.341672    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" event={"ID":"747c5890-78e9-41bf-b864-5496e87ec403","Type":"ContainerStarted","Data":"bdb671bb4577a0557e2486bb0a24d2b9300efc24af2b4c324c025789b642d6bd"}
Jul 17 18:06:40.341860 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:40.341724    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" event={"ID":"747c5890-78e9-41bf-b864-5496e87ec403","Type":"ContainerStarted","Data":"67dbe3238ee26a24a06e998eedbb35dd8e6d46e214ca9951ccf88da914352e09"}
Jul 17 18:06:40.396271 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:40.396209    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" podStartSLOduration=1.396190333 podStartE2EDuration="1.396190333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:06:40.395050362 +0000 UTC m=+2572.400406216" watchObservedRunningTime="2026-07-17 18:06:40.396190333 +0000 UTC m=+2572.401546184"
Jul 17 18:06:41.348087 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:41.348047    2572 generic.go:356] "Generic (PLEG): container finished" podID="747c5890-78e9-41bf-b864-5496e87ec403" containerID="bdb671bb4577a0557e2486bb0a24d2b9300efc24af2b4c324c025789b642d6bd" exitCode=6
Jul 17 18:06:41.348550 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:41.348139    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" event={"ID":"747c5890-78e9-41bf-b864-5496e87ec403","Type":"ContainerDied","Data":"bdb671bb4577a0557e2486bb0a24d2b9300efc24af2b4c324c025789b642d6bd"}
Jul 17 18:06:41.348550 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:41.348295    2572 scope.go:117] "RemoveContainer" containerID="bdb671bb4577a0557e2486bb0a24d2b9300efc24af2b4c324c025789b642d6bd"
Jul 17 18:06:42.355267 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:42.355224    2572 generic.go:356] "Generic (PLEG): container finished" podID="747c5890-78e9-41bf-b864-5496e87ec403" containerID="40f1a08de9bcfe3dea74107e22790bcb8f5a31ae8d370ccd29a880d06ec16b08" exitCode=6
Jul 17 18:06:42.355773 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:42.355274    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" event={"ID":"747c5890-78e9-41bf-b864-5496e87ec403","Type":"ContainerDied","Data":"40f1a08de9bcfe3dea74107e22790bcb8f5a31ae8d370ccd29a880d06ec16b08"}
Jul 17 18:06:42.355773 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:42.355336    2572 scope.go:117] "RemoveContainer" containerID="bdb671bb4577a0557e2486bb0a24d2b9300efc24af2b4c324c025789b642d6bd"
Jul 17 18:06:42.355773 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:42.355572    2572 scope.go:117] "RemoveContainer" containerID="40f1a08de9bcfe3dea74107e22790bcb8f5a31ae8d370ccd29a880d06ec16b08"
Jul 17 18:06:42.355916 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:06:42.355833    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-7wftf_odh-ai-gateway-infra(747c5890-78e9-41bf-b864-5496e87ec403)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" podUID="747c5890-78e9-41bf-b864-5496e87ec403"
Jul 17 18:06:43.362192 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:43.362156    2572 scope.go:117] "RemoveContainer" containerID="40f1a08de9bcfe3dea74107e22790bcb8f5a31ae8d370ccd29a880d06ec16b08"
Jul 17 18:06:43.362642 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:06:43.362381    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-7wftf_odh-ai-gateway-infra(747c5890-78e9-41bf-b864-5496e87ec403)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" podUID="747c5890-78e9-41bf-b864-5496e87ec403"
Jul 17 18:06:54.157686 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.157650    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"]
Jul 17 18:06:54.158189 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.157987    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerName="revoke-keys"
Jul 17 18:06:54.158189 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.157997    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerName="revoke-keys"
Jul 17 18:06:54.158189 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.158054    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b62292b-89c5-44a7-9028-619ac8c0f6a0" containerName="revoke-keys"
Jul 17 18:06:54.162636 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.162611    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"
Jul 17 18:06:54.168770 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.168743    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"]
Jul 17 18:06:54.183768 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.183731    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c986ba-0552-4411-9fa0-fcb873198c73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7\" (UID: \"f9c986ba-0552-4411-9fa0-fcb873198c73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"
Jul 17 18:06:54.284732 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.284695    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c986ba-0552-4411-9fa0-fcb873198c73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7\" (UID: \"f9c986ba-0552-4411-9fa0-fcb873198c73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"
Jul 17 18:06:54.285332 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.285310    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c986ba-0552-4411-9fa0-fcb873198c73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7\" (UID: \"f9c986ba-0552-4411-9fa0-fcb873198c73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"
Jul 17 18:06:54.475581 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.475435    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"
Jul 17 18:06:54.637919 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:54.637890    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"]
Jul 17 18:06:54.639273 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:06:54.639246    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9c986ba_0552_4411_9fa0_fcb873198c73.slice/crio-ef232b04d1e3ec872bff45c6b4492d10e9d4d6e1cdc8945685a1910b0bf4f410 WatchSource:0}: Error finding container ef232b04d1e3ec872bff45c6b4492d10e9d4d6e1cdc8945685a1910b0bf4f410: Status 404 returned error can't find the container with id ef232b04d1e3ec872bff45c6b4492d10e9d4d6e1cdc8945685a1910b0bf4f410
Jul 17 18:06:55.424424 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:55.424382    2572 generic.go:356] "Generic (PLEG): container finished" podID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerID="66275420f21ab1e1bae9ca4e46af17ace3391c58bc60251a6617651e17f0cea4" exitCode=6
Jul 17 18:06:55.425115 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:55.424436    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7" event={"ID":"f9c986ba-0552-4411-9fa0-fcb873198c73","Type":"ContainerDied","Data":"66275420f21ab1e1bae9ca4e46af17ace3391c58bc60251a6617651e17f0cea4"}
Jul 17 18:06:55.425115 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:55.424513    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7" event={"ID":"f9c986ba-0552-4411-9fa0-fcb873198c73","Type":"ContainerStarted","Data":"ef232b04d1e3ec872bff45c6b4492d10e9d4d6e1cdc8945685a1910b0bf4f410"}
Jul 17 18:06:55.425115 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:55.424740    2572 scope.go:117] "RemoveContainer" containerID="66275420f21ab1e1bae9ca4e46af17ace3391c58bc60251a6617651e17f0cea4"
Jul 17 18:06:56.438716 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:56.438623    2572 generic.go:356] "Generic (PLEG): container finished" podID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerID="9142d812283fefb2b225ee64c9ec98e6fc7339f9f91f57bd90f11c937acf44a0" exitCode=6
Jul 17 18:06:56.438716 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:56.438689    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7" event={"ID":"f9c986ba-0552-4411-9fa0-fcb873198c73","Type":"ContainerDied","Data":"9142d812283fefb2b225ee64c9ec98e6fc7339f9f91f57bd90f11c937acf44a0"}
Jul 17 18:06:56.439186 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:56.438765    2572 scope.go:117] "RemoveContainer" containerID="66275420f21ab1e1bae9ca4e46af17ace3391c58bc60251a6617651e17f0cea4"
Jul 17 18:06:56.439186 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:56.438977    2572 scope.go:117] "RemoveContainer" containerID="9142d812283fefb2b225ee64c9ec98e6fc7339f9f91f57bd90f11c937acf44a0"
Jul 17 18:06:56.439255 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:06:56.439224    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7_odh-ai-gateway-infra(f9c986ba-0552-4411-9fa0-fcb873198c73)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73"
Jul 17 18:06:57.446385 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:57.446355    2572 scope.go:117] "RemoveContainer" containerID="9142d812283fefb2b225ee64c9ec98e6fc7339f9f91f57bd90f11c937acf44a0"
Jul 17 18:06:57.446822 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:06:57.446580    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7_odh-ai-gateway-infra(f9c986ba-0552-4411-9fa0-fcb873198c73)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73"
Jul 17 18:06:58.512207 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:58.512163    2572 scope.go:117] "RemoveContainer" containerID="40f1a08de9bcfe3dea74107e22790bcb8f5a31ae8d370ccd29a880d06ec16b08"
Jul 17 18:06:59.459080 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:59.459043    2572 generic.go:356] "Generic (PLEG): container finished" podID="747c5890-78e9-41bf-b864-5496e87ec403" containerID="ae7449bf77ef540f8e3bebb59844f120f4df1c67e394e5df843646096f2dd79f" exitCode=6
Jul 17 18:06:59.459308 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:59.459116    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" event={"ID":"747c5890-78e9-41bf-b864-5496e87ec403","Type":"ContainerDied","Data":"ae7449bf77ef540f8e3bebb59844f120f4df1c67e394e5df843646096f2dd79f"}
Jul 17 18:06:59.459308 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:59.459179    2572 scope.go:117] "RemoveContainer" containerID="40f1a08de9bcfe3dea74107e22790bcb8f5a31ae8d370ccd29a880d06ec16b08"
Jul 17 18:06:59.459505 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:06:59.459472    2572 scope.go:117] "RemoveContainer" containerID="ae7449bf77ef540f8e3bebb59844f120f4df1c67e394e5df843646096f2dd79f"
Jul 17 18:06:59.459716 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:06:59.459696    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-7wftf_odh-ai-gateway-infra(747c5890-78e9-41bf-b864-5496e87ec403)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" podUID="747c5890-78e9-41bf-b864-5496e87ec403"
Jul 17 18:07:00.520134 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:00.520102    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"]
Jul 17 18:07:00.666200 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:00.666170    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"
Jul 17 18:07:00.744467 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:00.744422    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/747c5890-78e9-41bf-b864-5496e87ec403-maas-api-service-ca\") pod \"747c5890-78e9-41bf-b864-5496e87ec403\" (UID: \"747c5890-78e9-41bf-b864-5496e87ec403\") "
Jul 17 18:07:00.744865 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:00.744842    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/747c5890-78e9-41bf-b864-5496e87ec403-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "747c5890-78e9-41bf-b864-5496e87ec403" (UID: "747c5890-78e9-41bf-b864-5496e87ec403"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:07:00.845878 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:00.845841    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/747c5890-78e9-41bf-b864-5496e87ec403-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:07:01.472865 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:01.472835    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"
Jul 17 18:07:01.473052 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:01.472830    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf" event={"ID":"747c5890-78e9-41bf-b864-5496e87ec403","Type":"ContainerDied","Data":"67dbe3238ee26a24a06e998eedbb35dd8e6d46e214ca9951ccf88da914352e09"}
Jul 17 18:07:01.473052 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:01.473013    2572 scope.go:117] "RemoveContainer" containerID="ae7449bf77ef540f8e3bebb59844f120f4df1c67e394e5df843646096f2dd79f"
Jul 17 18:07:01.495852 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:01.495812    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"]
Jul 17 18:07:01.502380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:01.502346    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-7wftf"]
Jul 17 18:07:02.515561 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:02.515528    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="747c5890-78e9-41bf-b864-5496e87ec403" path="/var/lib/kubelet/pods/747c5890-78e9-41bf-b864-5496e87ec403/volumes"
Jul 17 18:07:12.510135 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:12.510048    2572 scope.go:117] "RemoveContainer" containerID="9142d812283fefb2b225ee64c9ec98e6fc7339f9f91f57bd90f11c937acf44a0"
Jul 17 18:07:13.543774 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:13.543682    2572 generic.go:356] "Generic (PLEG): container finished" podID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerID="b6a91374f8e96b3f49e15145a3b106fdddadcce4a5570e941e8bd0e8e751b341" exitCode=6
Jul 17 18:07:13.543774 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:13.543757    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7" event={"ID":"f9c986ba-0552-4411-9fa0-fcb873198c73","Type":"ContainerDied","Data":"b6a91374f8e96b3f49e15145a3b106fdddadcce4a5570e941e8bd0e8e751b341"}
Jul 17 18:07:13.544243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:13.543820    2572 scope.go:117] "RemoveContainer" containerID="9142d812283fefb2b225ee64c9ec98e6fc7339f9f91f57bd90f11c937acf44a0"
Jul 17 18:07:13.544243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:13.544108    2572 scope.go:117] "RemoveContainer" containerID="b6a91374f8e96b3f49e15145a3b106fdddadcce4a5570e941e8bd0e8e751b341"
Jul 17 18:07:13.544347 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:07:13.544323    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7_odh-ai-gateway-infra(f9c986ba-0552-4411-9fa0-fcb873198c73)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73"
Jul 17 18:07:14.585948 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:14.585908    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"]
Jul 17 18:07:14.731805 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:14.731774    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"
Jul 17 18:07:14.875250 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:14.875137    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c986ba-0552-4411-9fa0-fcb873198c73-maas-api-service-ca\") pod \"f9c986ba-0552-4411-9fa0-fcb873198c73\" (UID: \"f9c986ba-0552-4411-9fa0-fcb873198c73\") "
Jul 17 18:07:14.875576 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:14.875547    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9c986ba-0552-4411-9fa0-fcb873198c73-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f9c986ba-0552-4411-9fa0-fcb873198c73" (UID: "f9c986ba-0552-4411-9fa0-fcb873198c73"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:07:14.975983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:14.975939    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c986ba-0552-4411-9fa0-fcb873198c73-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:07:15.557047 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:15.557016    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"
Jul 17 18:07:15.557284 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:15.557010    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7" event={"ID":"f9c986ba-0552-4411-9fa0-fcb873198c73","Type":"ContainerDied","Data":"ef232b04d1e3ec872bff45c6b4492d10e9d4d6e1cdc8945685a1910b0bf4f410"}
Jul 17 18:07:15.557284 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:15.557153    2572 scope.go:117] "RemoveContainer" containerID="b6a91374f8e96b3f49e15145a3b106fdddadcce4a5570e941e8bd0e8e751b341"
Jul 17 18:07:15.581043 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:15.581003    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"]
Jul 17 18:07:15.589536 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:15.589475    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-ppxh7"]
Jul 17 18:07:16.515214 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:16.515176    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" path="/var/lib/kubelet/pods/f9c986ba-0552-4411-9fa0-fcb873198c73/volumes"
Jul 17 18:07:29.137085 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137047    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"]
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137398    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747c5890-78e9-41bf-b864-5496e87ec403" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137410    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="747c5890-78e9-41bf-b864-5496e87ec403" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137423    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137429    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137437    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137442    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137448    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747c5890-78e9-41bf-b864-5496e87ec403" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137453    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="747c5890-78e9-41bf-b864-5496e87ec403" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137462    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137467    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137542    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137551    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="747c5890-78e9-41bf-b864-5496e87ec403" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137557    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="747c5890-78e9-41bf-b864-5496e87ec403" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137563    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="747c5890-78e9-41bf-b864-5496e87ec403" containerName="revoke-keys"
Jul 17 18:07:29.137630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.137571    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerName="revoke-keys"
Jul 17 18:07:29.140448 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.140431    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"
Jul 17 18:07:29.143765 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.143738    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:07:29.143765 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.143759    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:07:29.150297 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.150179    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"]
Jul 17 18:07:29.210332 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.210271    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aac9f26-97d1-434d-867b-09e12388fd33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj\" (UID: \"9aac9f26-97d1-434d-867b-09e12388fd33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"
Jul 17 18:07:29.311154 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.311112    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aac9f26-97d1-434d-867b-09e12388fd33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj\" (UID: \"9aac9f26-97d1-434d-867b-09e12388fd33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"
Jul 17 18:07:29.311881 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.311860    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aac9f26-97d1-434d-867b-09e12388fd33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj\" (UID: \"9aac9f26-97d1-434d-867b-09e12388fd33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"
Jul 17 18:07:29.453018 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.452921    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"
Jul 17 18:07:29.627130 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:29.627098    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"]
Jul 17 18:07:30.637615 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:30.637520    2572 generic.go:356] "Generic (PLEG): container finished" podID="9aac9f26-97d1-434d-867b-09e12388fd33" containerID="42003d22fb8a03c68b66840be329b4ba8163d880abc8c5fd8f0a15c2e6a9a09e" exitCode=0
Jul 17 18:07:30.637988 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:30.637608    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj" event={"ID":"9aac9f26-97d1-434d-867b-09e12388fd33","Type":"ContainerDied","Data":"42003d22fb8a03c68b66840be329b4ba8163d880abc8c5fd8f0a15c2e6a9a09e"}
Jul 17 18:07:30.637988 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:30.637659    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj" event={"ID":"9aac9f26-97d1-434d-867b-09e12388fd33","Type":"ContainerStarted","Data":"053493e84b1247d75de71e2b51eed255606db43a36bbdf5d867ba2f4f45009b7"}
Jul 17 18:07:31.786362 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:31.786330    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"
Jul 17 18:07:31.834132 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:31.834089    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aac9f26-97d1-434d-867b-09e12388fd33-maas-api-service-ca\") pod \"9aac9f26-97d1-434d-867b-09e12388fd33\" (UID: \"9aac9f26-97d1-434d-867b-09e12388fd33\") "
Jul 17 18:07:31.834556 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:31.834530    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aac9f26-97d1-434d-867b-09e12388fd33-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9aac9f26-97d1-434d-867b-09e12388fd33" (UID: "9aac9f26-97d1-434d-867b-09e12388fd33"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:07:31.935770 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:31.935668    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aac9f26-97d1-434d-867b-09e12388fd33-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:07:32.649475 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:32.649439    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj"
Jul 17 18:07:32.649675 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:32.649473    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj" event={"ID":"9aac9f26-97d1-434d-867b-09e12388fd33","Type":"ContainerDied","Data":"053493e84b1247d75de71e2b51eed255606db43a36bbdf5d867ba2f4f45009b7"}
Jul 17 18:07:32.649675 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:32.649534    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="053493e84b1247d75de71e2b51eed255606db43a36bbdf5d867ba2f4f45009b7"
Jul 17 18:07:39.685788 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.685750    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"]
Jul 17 18:07:39.686184 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.686072    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747c5890-78e9-41bf-b864-5496e87ec403" containerName="revoke-keys"
Jul 17 18:07:39.686184 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.686082    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="747c5890-78e9-41bf-b864-5496e87ec403" containerName="revoke-keys"
Jul 17 18:07:39.686184 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.686092    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aac9f26-97d1-434d-867b-09e12388fd33" containerName="revoke-keys"
Jul 17 18:07:39.686184 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.686098    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aac9f26-97d1-434d-867b-09e12388fd33" containerName="revoke-keys"
Jul 17 18:07:39.686184 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.686154    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9c986ba-0552-4411-9fa0-fcb873198c73" containerName="revoke-keys"
Jul 17 18:07:39.686184 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.686163    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aac9f26-97d1-434d-867b-09e12388fd33" containerName="revoke-keys"
Jul 17 18:07:39.689249 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.689228    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"
Jul 17 18:07:39.692551 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.692529    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:07:39.692670 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.692531    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:07:39.705330 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.705301    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"]
Jul 17 18:07:39.801982 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.801937    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff78500-eb0a-4f24-80c7-1da3addd5507-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f\" (UID: \"fff78500-eb0a-4f24-80c7-1da3addd5507\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"
Jul 17 18:07:39.903434 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.903394    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff78500-eb0a-4f24-80c7-1da3addd5507-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f\" (UID: \"fff78500-eb0a-4f24-80c7-1da3addd5507\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"
Jul 17 18:07:39.907771 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.904076    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff78500-eb0a-4f24-80c7-1da3addd5507-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f\" (UID: \"fff78500-eb0a-4f24-80c7-1da3addd5507\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"
Jul 17 18:07:39.999714 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:39.999606    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"
Jul 17 18:07:40.161354 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:40.161324    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"]
Jul 17 18:07:40.162952 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:07:40.162923    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff78500_eb0a_4f24_80c7_1da3addd5507.slice/crio-50d27b1c7c4e750cefe364a18082a449f8b7af01e87bbf5516d2faa8ad605aca WatchSource:0}: Error finding container 50d27b1c7c4e750cefe364a18082a449f8b7af01e87bbf5516d2faa8ad605aca: Status 404 returned error can't find the container with id 50d27b1c7c4e750cefe364a18082a449f8b7af01e87bbf5516d2faa8ad605aca
Jul 17 18:07:40.696216 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:40.696112    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" event={"ID":"fff78500-eb0a-4f24-80c7-1da3addd5507","Type":"ContainerStarted","Data":"90dc9afc492853e1391425e0e935dcc7d5088c55f32e0230337bfdbbf8ff14f7"}
Jul 17 18:07:40.696216 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:40.696172    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" event={"ID":"fff78500-eb0a-4f24-80c7-1da3addd5507","Type":"ContainerStarted","Data":"50d27b1c7c4e750cefe364a18082a449f8b7af01e87bbf5516d2faa8ad605aca"}
Jul 17 18:07:40.714217 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:40.714157    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" podStartSLOduration=1.714139207 podStartE2EDuration="1.714139207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:07:40.713634752 +0000 UTC m=+2632.718990621" watchObservedRunningTime="2026-07-17 18:07:40.714139207 +0000 UTC m=+2632.719495118"
Jul 17 18:07:41.703379 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:41.703343    2572 generic.go:356] "Generic (PLEG): container finished" podID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerID="90dc9afc492853e1391425e0e935dcc7d5088c55f32e0230337bfdbbf8ff14f7" exitCode=6
Jul 17 18:07:41.703815 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:41.703404    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" event={"ID":"fff78500-eb0a-4f24-80c7-1da3addd5507","Type":"ContainerDied","Data":"90dc9afc492853e1391425e0e935dcc7d5088c55f32e0230337bfdbbf8ff14f7"}
Jul 17 18:07:41.703815 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:41.703620    2572 scope.go:117] "RemoveContainer" containerID="90dc9afc492853e1391425e0e935dcc7d5088c55f32e0230337bfdbbf8ff14f7"
Jul 17 18:07:42.710531 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:42.710399    2572 generic.go:356] "Generic (PLEG): container finished" podID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerID="bf2e42b20e4a5359f3124c8abd14184c16264fd9910e5d913c1a6a0e7c5576e4" exitCode=6
Jul 17 18:07:42.711009 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:42.710518    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" event={"ID":"fff78500-eb0a-4f24-80c7-1da3addd5507","Type":"ContainerDied","Data":"bf2e42b20e4a5359f3124c8abd14184c16264fd9910e5d913c1a6a0e7c5576e4"}
Jul 17 18:07:42.711009 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:42.710593    2572 scope.go:117] "RemoveContainer" containerID="90dc9afc492853e1391425e0e935dcc7d5088c55f32e0230337bfdbbf8ff14f7"
Jul 17 18:07:42.711009 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:42.710809    2572 scope.go:117] "RemoveContainer" containerID="bf2e42b20e4a5359f3124c8abd14184c16264fd9910e5d913c1a6a0e7c5576e4"
Jul 17 18:07:42.711137 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:07:42.711012    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f_odh-ai-gateway-infra(fff78500-eb0a-4f24-80c7-1da3addd5507)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507"
Jul 17 18:07:43.717731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:43.717700    2572 scope.go:117] "RemoveContainer" containerID="bf2e42b20e4a5359f3124c8abd14184c16264fd9910e5d913c1a6a0e7c5576e4"
Jul 17 18:07:43.718121 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:07:43.717928    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f_odh-ai-gateway-infra(fff78500-eb0a-4f24-80c7-1da3addd5507)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507"
Jul 17 18:07:54.222636 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:54.222599    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"]
Jul 17 18:07:54.227152 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:54.227130    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"
Jul 17 18:07:54.235620 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:54.235588    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"]
Jul 17 18:07:54.328906 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:54.328856    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0434e8e3-7076-480f-8434-51757f6991ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc\" (UID: \"0434e8e3-7076-480f-8434-51757f6991ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"
Jul 17 18:07:54.430355 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:54.430318    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0434e8e3-7076-480f-8434-51757f6991ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc\" (UID: \"0434e8e3-7076-480f-8434-51757f6991ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"
Jul 17 18:07:54.430990 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:54.430968    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0434e8e3-7076-480f-8434-51757f6991ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc\" (UID: \"0434e8e3-7076-480f-8434-51757f6991ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"
Jul 17 18:07:54.539023 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:54.538917    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"
Jul 17 18:07:54.682386 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:54.682351    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"]
Jul 17 18:07:54.684475 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:07:54.684443    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0434e8e3_7076_480f_8434_51757f6991ed.slice/crio-eeae7232c6728985dc9d4eaf687c17b3a68b672579867a25a271378a32b078f3 WatchSource:0}: Error finding container eeae7232c6728985dc9d4eaf687c17b3a68b672579867a25a271378a32b078f3: Status 404 returned error can't find the container with id eeae7232c6728985dc9d4eaf687c17b3a68b672579867a25a271378a32b078f3
Jul 17 18:07:54.781922 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:54.781886    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" event={"ID":"0434e8e3-7076-480f-8434-51757f6991ed","Type":"ContainerStarted","Data":"eeae7232c6728985dc9d4eaf687c17b3a68b672579867a25a271378a32b078f3"}
Jul 17 18:07:55.788750 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:55.788645    2572 generic.go:356] "Generic (PLEG): container finished" podID="0434e8e3-7076-480f-8434-51757f6991ed" containerID="bccb4bcd80102adb908ca7c226a3530767d0ee5d2fdc0ba9965f46eca85d6271" exitCode=6
Jul 17 18:07:55.788750 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:55.788736    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" event={"ID":"0434e8e3-7076-480f-8434-51757f6991ed","Type":"ContainerDied","Data":"bccb4bcd80102adb908ca7c226a3530767d0ee5d2fdc0ba9965f46eca85d6271"}
Jul 17 18:07:55.789197 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:55.788929    2572 scope.go:117] "RemoveContainer" containerID="bccb4bcd80102adb908ca7c226a3530767d0ee5d2fdc0ba9965f46eca85d6271"
Jul 17 18:07:56.797130 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:56.797030    2572 generic.go:356] "Generic (PLEG): container finished" podID="0434e8e3-7076-480f-8434-51757f6991ed" containerID="e05d6b3e9cf41bf386318465e305240e4ee63676d27b96d772203c7a29452b40" exitCode=6
Jul 17 18:07:56.797130 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:56.797113    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" event={"ID":"0434e8e3-7076-480f-8434-51757f6991ed","Type":"ContainerDied","Data":"e05d6b3e9cf41bf386318465e305240e4ee63676d27b96d772203c7a29452b40"}
Jul 17 18:07:56.797696 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:56.797161    2572 scope.go:117] "RemoveContainer" containerID="bccb4bcd80102adb908ca7c226a3530767d0ee5d2fdc0ba9965f46eca85d6271"
Jul 17 18:07:56.797696 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:56.797340    2572 scope.go:117] "RemoveContainer" containerID="e05d6b3e9cf41bf386318465e305240e4ee63676d27b96d772203c7a29452b40"
Jul 17 18:07:56.797696 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:07:56.797641    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc_odh-ai-gateway-infra(0434e8e3-7076-480f-8434-51757f6991ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" podUID="0434e8e3-7076-480f-8434-51757f6991ed"
Jul 17 18:07:57.510123 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:57.510087    2572 scope.go:117] "RemoveContainer" containerID="bf2e42b20e4a5359f3124c8abd14184c16264fd9910e5d913c1a6a0e7c5576e4"
Jul 17 18:07:57.803927 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:57.803845    2572 scope.go:117] "RemoveContainer" containerID="e05d6b3e9cf41bf386318465e305240e4ee63676d27b96d772203c7a29452b40"
Jul 17 18:07:57.804298 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:07:57.804052    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc_odh-ai-gateway-infra(0434e8e3-7076-480f-8434-51757f6991ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" podUID="0434e8e3-7076-480f-8434-51757f6991ed"
Jul 17 18:07:58.089352 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.089317    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"]
Jul 17 18:07:58.092833 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.092813    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"
Jul 17 18:07:58.104503 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.104458    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"]
Jul 17 18:07:58.167337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.167299    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcfb2453-d382-4a30-831e-8552837c59bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2\" (UID: \"fcfb2453-d382-4a30-831e-8552837c59bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"
Jul 17 18:07:58.268589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.268547    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcfb2453-d382-4a30-831e-8552837c59bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2\" (UID: \"fcfb2453-d382-4a30-831e-8552837c59bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"
Jul 17 18:07:58.269186 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.269166    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcfb2453-d382-4a30-831e-8552837c59bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2\" (UID: \"fcfb2453-d382-4a30-831e-8552837c59bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"
Jul 17 18:07:58.404257 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.404152    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"
Jul 17 18:07:58.547775 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.547739    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"]
Jul 17 18:07:58.810514 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.810457    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" event={"ID":"fcfb2453-d382-4a30-831e-8552837c59bf","Type":"ContainerStarted","Data":"b96a3d50c173703364266a59785c17cd09057412be39979ea20f210794e69a39"}
Jul 17 18:07:58.812409 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.812373    2572 generic.go:356] "Generic (PLEG): container finished" podID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerID="593414f608f491da8ce57b7e12043bc40e50f836add7190723f11cfb6605d565" exitCode=6
Jul 17 18:07:58.812571 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.812448    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" event={"ID":"fff78500-eb0a-4f24-80c7-1da3addd5507","Type":"ContainerDied","Data":"593414f608f491da8ce57b7e12043bc40e50f836add7190723f11cfb6605d565"}
Jul 17 18:07:58.812571 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.812531    2572 scope.go:117] "RemoveContainer" containerID="bf2e42b20e4a5359f3124c8abd14184c16264fd9910e5d913c1a6a0e7c5576e4"
Jul 17 18:07:58.812847 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:58.812828    2572 scope.go:117] "RemoveContainer" containerID="593414f608f491da8ce57b7e12043bc40e50f836add7190723f11cfb6605d565"
Jul 17 18:07:58.813080 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:07:58.813055    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f_odh-ai-gateway-infra(fff78500-eb0a-4f24-80c7-1da3addd5507)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507"
Jul 17 18:07:59.820695 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:59.820656    2572 generic.go:356] "Generic (PLEG): container finished" podID="fcfb2453-d382-4a30-831e-8552837c59bf" containerID="3363f0695c5430b7093dd7b510a59a5e51acf1e63a39a68f058ba14bc96e5738" exitCode=6
Jul 17 18:07:59.821141 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:59.820750    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" event={"ID":"fcfb2453-d382-4a30-831e-8552837c59bf","Type":"ContainerDied","Data":"3363f0695c5430b7093dd7b510a59a5e51acf1e63a39a68f058ba14bc96e5738"}
Jul 17 18:07:59.821141 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:59.820900    2572 scope.go:117] "RemoveContainer" containerID="3363f0695c5430b7093dd7b510a59a5e51acf1e63a39a68f058ba14bc96e5738"
Jul 17 18:07:59.840188 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:59.840154    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"]
Jul 17 18:07:59.987782 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:07:59.987751    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"
Jul 17 18:08:00.086960 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.086864    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff78500-eb0a-4f24-80c7-1da3addd5507-maas-api-service-ca\") pod \"fff78500-eb0a-4f24-80c7-1da3addd5507\" (UID: \"fff78500-eb0a-4f24-80c7-1da3addd5507\") "
Jul 17 18:08:00.087226 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.087202    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fff78500-eb0a-4f24-80c7-1da3addd5507-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fff78500-eb0a-4f24-80c7-1da3addd5507" (UID: "fff78500-eb0a-4f24-80c7-1da3addd5507"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:08:00.187747 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.187710    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff78500-eb0a-4f24-80c7-1da3addd5507-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:08:00.827923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.827894    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"
Jul 17 18:08:00.828441 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.827934    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f" event={"ID":"fff78500-eb0a-4f24-80c7-1da3addd5507","Type":"ContainerDied","Data":"50d27b1c7c4e750cefe364a18082a449f8b7af01e87bbf5516d2faa8ad605aca"}
Jul 17 18:08:00.828441 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.828002    2572 scope.go:117] "RemoveContainer" containerID="593414f608f491da8ce57b7e12043bc40e50f836add7190723f11cfb6605d565"
Jul 17 18:08:00.830154 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.830125    2572 generic.go:356] "Generic (PLEG): container finished" podID="fcfb2453-d382-4a30-831e-8552837c59bf" containerID="4ce31621bf9ac22208ad346899fb2d1d22b4c1994c40b2f929ecf2766add9934" exitCode=6
Jul 17 18:08:00.830440 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.830219    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" event={"ID":"fcfb2453-d382-4a30-831e-8552837c59bf","Type":"ContainerDied","Data":"4ce31621bf9ac22208ad346899fb2d1d22b4c1994c40b2f929ecf2766add9934"}
Jul 17 18:08:00.830553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.830479    2572 scope.go:117] "RemoveContainer" containerID="4ce31621bf9ac22208ad346899fb2d1d22b4c1994c40b2f929ecf2766add9934"
Jul 17 18:08:00.830772 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:00.830746    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2_odh-ai-gateway-infra(fcfb2453-d382-4a30-831e-8552837c59bf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" podUID="fcfb2453-d382-4a30-831e-8552837c59bf"
Jul 17 18:08:00.837476 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.837456    2572 scope.go:117] "RemoveContainer" containerID="3363f0695c5430b7093dd7b510a59a5e51acf1e63a39a68f058ba14bc96e5738"
Jul 17 18:08:00.844812 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.844779    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"]
Jul 17 18:08:00.849203 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:00.849176    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-sbf5f"]
Jul 17 18:08:01.837589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:01.837559    2572 scope.go:117] "RemoveContainer" containerID="4ce31621bf9ac22208ad346899fb2d1d22b4c1994c40b2f929ecf2766add9934"
Jul 17 18:08:01.838067 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:01.837784    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2_odh-ai-gateway-infra(fcfb2453-d382-4a30-831e-8552837c59bf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" podUID="fcfb2453-d382-4a30-831e-8552837c59bf"
Jul 17 18:08:02.515243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:02.515201    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" path="/var/lib/kubelet/pods/fff78500-eb0a-4f24-80c7-1da3addd5507/volumes"
Jul 17 18:08:06.537118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537082    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"]
Jul 17 18:08:06.537610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537467    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerName="revoke-keys"
Jul 17 18:08:06.537610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537482    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerName="revoke-keys"
Jul 17 18:08:06.537610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537510    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerName="revoke-keys"
Jul 17 18:08:06.537610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537516    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerName="revoke-keys"
Jul 17 18:08:06.537610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537526    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerName="revoke-keys"
Jul 17 18:08:06.537610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537531    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerName="revoke-keys"
Jul 17 18:08:06.537610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537609    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerName="revoke-keys"
Jul 17 18:08:06.537883 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537620    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerName="revoke-keys"
Jul 17 18:08:06.537883 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.537626    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fff78500-eb0a-4f24-80c7-1da3addd5507" containerName="revoke-keys"
Jul 17 18:08:06.541932 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.541909    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"
Jul 17 18:08:06.548507 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.548432    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"]
Jul 17 18:08:06.645390 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.645353    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94a8903-6da1-4e3e-ba03-4c81a943eacd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs\" (UID: \"f94a8903-6da1-4e3e-ba03-4c81a943eacd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"
Jul 17 18:08:06.746849 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.746805    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94a8903-6da1-4e3e-ba03-4c81a943eacd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs\" (UID: \"f94a8903-6da1-4e3e-ba03-4c81a943eacd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"
Jul 17 18:08:06.747470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.747449    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94a8903-6da1-4e3e-ba03-4c81a943eacd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs\" (UID: \"f94a8903-6da1-4e3e-ba03-4c81a943eacd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"
Jul 17 18:08:06.854416 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:06.854364    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"
Jul 17 18:08:07.010223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:07.010192    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"]
Jul 17 18:08:07.012172 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:08:07.012134    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94a8903_6da1_4e3e_ba03_4c81a943eacd.slice/crio-810d796cccd209a15fb242a7ca3f3330b4807807601b45dac53f8b3fa7564172 WatchSource:0}: Error finding container 810d796cccd209a15fb242a7ca3f3330b4807807601b45dac53f8b3fa7564172: Status 404 returned error can't find the container with id 810d796cccd209a15fb242a7ca3f3330b4807807601b45dac53f8b3fa7564172
Jul 17 18:08:07.884176 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:07.884072    2572 generic.go:356] "Generic (PLEG): container finished" podID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerID="d28640bf4f8cb8e963bd67b411b05d4a285901741a1d56775ae0f24d30053c7f" exitCode=6
Jul 17 18:08:07.884176 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:07.884158    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" event={"ID":"f94a8903-6da1-4e3e-ba03-4c81a943eacd","Type":"ContainerDied","Data":"d28640bf4f8cb8e963bd67b411b05d4a285901741a1d56775ae0f24d30053c7f"}
Jul 17 18:08:07.884651 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:07.884222    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" event={"ID":"f94a8903-6da1-4e3e-ba03-4c81a943eacd","Type":"ContainerStarted","Data":"810d796cccd209a15fb242a7ca3f3330b4807807601b45dac53f8b3fa7564172"}
Jul 17 18:08:07.884651 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:07.884357    2572 scope.go:117] "RemoveContainer" containerID="d28640bf4f8cb8e963bd67b411b05d4a285901741a1d56775ae0f24d30053c7f"
Jul 17 18:08:08.891896 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:08.891860    2572 generic.go:356] "Generic (PLEG): container finished" podID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerID="1fece4959f41f28616d5d140bd47b152fc1e0db7d6758ce35c40bb91a527ef1b" exitCode=6
Jul 17 18:08:08.892337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:08.891945    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" event={"ID":"f94a8903-6da1-4e3e-ba03-4c81a943eacd","Type":"ContainerDied","Data":"1fece4959f41f28616d5d140bd47b152fc1e0db7d6758ce35c40bb91a527ef1b"}
Jul 17 18:08:08.892337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:08.892003    2572 scope.go:117] "RemoveContainer" containerID="d28640bf4f8cb8e963bd67b411b05d4a285901741a1d56775ae0f24d30053c7f"
Jul 17 18:08:08.892337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:08.892145    2572 scope.go:117] "RemoveContainer" containerID="1fece4959f41f28616d5d140bd47b152fc1e0db7d6758ce35c40bb91a527ef1b"
Jul 17 18:08:08.892483 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:08.892351    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs_odh-ai-gateway-infra(f94a8903-6da1-4e3e-ba03-4c81a943eacd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd"
Jul 17 18:08:09.510251 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:09.510217    2572 scope.go:117] "RemoveContainer" containerID="e05d6b3e9cf41bf386318465e305240e4ee63676d27b96d772203c7a29452b40"
Jul 17 18:08:09.900112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:09.900073    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" event={"ID":"0434e8e3-7076-480f-8434-51757f6991ed","Type":"ContainerStarted","Data":"84dbd4a6a9c7cab555107acea4597fd6fa56d94d13271169260425a44a023fe9"}
Jul 17 18:08:09.901947 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:09.901926    2572 scope.go:117] "RemoveContainer" containerID="1fece4959f41f28616d5d140bd47b152fc1e0db7d6758ce35c40bb91a527ef1b"
Jul 17 18:08:09.902130 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:09.902111    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs_odh-ai-gateway-infra(f94a8903-6da1-4e3e-ba03-4c81a943eacd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd"
Jul 17 18:08:09.945254 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:09.945201    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" podStartSLOduration=15.94518227 podStartE2EDuration="15.94518227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:08:09.943015876 +0000 UTC m=+2661.948371732" watchObservedRunningTime="2026-07-17 18:08:09.94518227 +0000 UTC m=+2661.950538124"
Jul 17 18:08:10.909867 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:10.909835    2572 generic.go:356] "Generic (PLEG): container finished" podID="0434e8e3-7076-480f-8434-51757f6991ed" containerID="84dbd4a6a9c7cab555107acea4597fd6fa56d94d13271169260425a44a023fe9" exitCode=6
Jul 17 18:08:10.910332 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:10.909908    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" event={"ID":"0434e8e3-7076-480f-8434-51757f6991ed","Type":"ContainerDied","Data":"84dbd4a6a9c7cab555107acea4597fd6fa56d94d13271169260425a44a023fe9"}
Jul 17 18:08:10.910332 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:10.909973    2572 scope.go:117] "RemoveContainer" containerID="e05d6b3e9cf41bf386318465e305240e4ee63676d27b96d772203c7a29452b40"
Jul 17 18:08:10.910332 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:10.910292    2572 scope.go:117] "RemoveContainer" containerID="84dbd4a6a9c7cab555107acea4597fd6fa56d94d13271169260425a44a023fe9"
Jul 17 18:08:10.910568 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:10.910544    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc_odh-ai-gateway-infra(0434e8e3-7076-480f-8434-51757f6991ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" podUID="0434e8e3-7076-480f-8434-51757f6991ed"
Jul 17 18:08:10.972106 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:10.972064    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"]
Jul 17 18:08:12.084469 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:12.084433    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"
Jul 17 18:08:12.194372 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:12.194231    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0434e8e3-7076-480f-8434-51757f6991ed-maas-api-service-ca\") pod \"0434e8e3-7076-480f-8434-51757f6991ed\" (UID: \"0434e8e3-7076-480f-8434-51757f6991ed\") "
Jul 17 18:08:12.194662 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:12.194637    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0434e8e3-7076-480f-8434-51757f6991ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0434e8e3-7076-480f-8434-51757f6991ed" (UID: "0434e8e3-7076-480f-8434-51757f6991ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:08:12.294937 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:12.294894    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0434e8e3-7076-480f-8434-51757f6991ed-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:08:12.924546 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:12.924506    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"
Jul 17 18:08:12.924546 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:12.924539    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc" event={"ID":"0434e8e3-7076-480f-8434-51757f6991ed","Type":"ContainerDied","Data":"eeae7232c6728985dc9d4eaf687c17b3a68b672579867a25a271378a32b078f3"}
Jul 17 18:08:12.924841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:12.924604    2572 scope.go:117] "RemoveContainer" containerID="84dbd4a6a9c7cab555107acea4597fd6fa56d94d13271169260425a44a023fe9"
Jul 17 18:08:12.945667 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:12.945631    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"]
Jul 17 18:08:12.950471 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:12.950427    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-7nccc"]
Jul 17 18:08:14.516185 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:14.516142    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0434e8e3-7076-480f-8434-51757f6991ed" path="/var/lib/kubelet/pods/0434e8e3-7076-480f-8434-51757f6991ed/volumes"
Jul 17 18:08:15.510178 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:15.510140    2572 scope.go:117] "RemoveContainer" containerID="4ce31621bf9ac22208ad346899fb2d1d22b4c1994c40b2f929ecf2766add9934"
Jul 17 18:08:15.947117 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:15.947079    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" event={"ID":"fcfb2453-d382-4a30-831e-8552837c59bf","Type":"ContainerStarted","Data":"b44194b33e53937f0a894f78d53de06b1e7c888dbd4afb1004d96839a217d6f5"}
Jul 17 18:08:15.965570 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:15.965508    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" podStartSLOduration=17.96547362 podStartE2EDuration="17.96547362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:07:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:08:15.964339348 +0000 UTC m=+2667.969695205" watchObservedRunningTime="2026-07-17 18:08:15.96547362 +0000 UTC m=+2667.970829477"
Jul 17 18:08:16.954898 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:16.954863    2572 generic.go:356] "Generic (PLEG): container finished" podID="fcfb2453-d382-4a30-831e-8552837c59bf" containerID="b44194b33e53937f0a894f78d53de06b1e7c888dbd4afb1004d96839a217d6f5" exitCode=6
Jul 17 18:08:16.955396 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:16.954940    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" event={"ID":"fcfb2453-d382-4a30-831e-8552837c59bf","Type":"ContainerDied","Data":"b44194b33e53937f0a894f78d53de06b1e7c888dbd4afb1004d96839a217d6f5"}
Jul 17 18:08:16.955396 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:16.955017    2572 scope.go:117] "RemoveContainer" containerID="4ce31621bf9ac22208ad346899fb2d1d22b4c1994c40b2f929ecf2766add9934"
Jul 17 18:08:16.955396 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:16.955311    2572 scope.go:117] "RemoveContainer" containerID="b44194b33e53937f0a894f78d53de06b1e7c888dbd4afb1004d96839a217d6f5"
Jul 17 18:08:16.955602 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:16.955532    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2_odh-ai-gateway-infra(fcfb2453-d382-4a30-831e-8552837c59bf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" podUID="fcfb2453-d382-4a30-831e-8552837c59bf"
Jul 17 18:08:16.989790 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:16.989745    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"]
Jul 17 18:08:18.107057 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:18.107030    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"
Jul 17 18:08:18.139858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:18.139819    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcfb2453-d382-4a30-831e-8552837c59bf-maas-api-service-ca\") pod \"fcfb2453-d382-4a30-831e-8552837c59bf\" (UID: \"fcfb2453-d382-4a30-831e-8552837c59bf\") "
Jul 17 18:08:18.140416 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:18.140227    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcfb2453-d382-4a30-831e-8552837c59bf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fcfb2453-d382-4a30-831e-8552837c59bf" (UID: "fcfb2453-d382-4a30-831e-8552837c59bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:08:18.240967 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:18.240872    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcfb2453-d382-4a30-831e-8552837c59bf-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:08:18.969951 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:18.969917    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"
Jul 17 18:08:18.970188 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:18.969940    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2" event={"ID":"fcfb2453-d382-4a30-831e-8552837c59bf","Type":"ContainerDied","Data":"b96a3d50c173703364266a59785c17cd09057412be39979ea20f210794e69a39"}
Jul 17 18:08:18.970188 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:18.970024    2572 scope.go:117] "RemoveContainer" containerID="b44194b33e53937f0a894f78d53de06b1e7c888dbd4afb1004d96839a217d6f5"
Jul 17 18:08:18.999877 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:18.999836    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"]
Jul 17 18:08:19.005743 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:19.005708    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-8s9f2"]
Jul 17 18:08:20.516380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:20.516348    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" path="/var/lib/kubelet/pods/fcfb2453-d382-4a30-831e-8552837c59bf/volumes"
Jul 17 18:08:21.510414 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:21.510372    2572 scope.go:117] "RemoveContainer" containerID="1fece4959f41f28616d5d140bd47b152fc1e0db7d6758ce35c40bb91a527ef1b"
Jul 17 18:08:21.990456 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:21.990421    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" event={"ID":"f94a8903-6da1-4e3e-ba03-4c81a943eacd","Type":"ContainerStarted","Data":"5e0e325df83760ca9370c9d463ac6827b297c2573f99b631caafdad865d147e1"}
Jul 17 18:08:22.008998 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:22.008932    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" podStartSLOduration=16.00891322 podStartE2EDuration="16.00891322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:08:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:08:22.007079192 +0000 UTC m=+2674.012435050" watchObservedRunningTime="2026-07-17 18:08:22.00891322 +0000 UTC m=+2674.014269074"
Jul 17 18:08:22.998202 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:22.998163    2572 generic.go:356] "Generic (PLEG): container finished" podID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerID="5e0e325df83760ca9370c9d463ac6827b297c2573f99b631caafdad865d147e1" exitCode=6
Jul 17 18:08:22.998681 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:22.998214    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" event={"ID":"f94a8903-6da1-4e3e-ba03-4c81a943eacd","Type":"ContainerDied","Data":"5e0e325df83760ca9370c9d463ac6827b297c2573f99b631caafdad865d147e1"}
Jul 17 18:08:22.998681 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:22.998297    2572 scope.go:117] "RemoveContainer" containerID="1fece4959f41f28616d5d140bd47b152fc1e0db7d6758ce35c40bb91a527ef1b"
Jul 17 18:08:22.998776 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:22.998689    2572 scope.go:117] "RemoveContainer" containerID="5e0e325df83760ca9370c9d463ac6827b297c2573f99b631caafdad865d147e1"
Jul 17 18:08:22.998920 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:22.998899    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs_odh-ai-gateway-infra(f94a8903-6da1-4e3e-ba03-4c81a943eacd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd"
Jul 17 18:08:23.025354 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:23.025307    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"]
Jul 17 18:08:24.145949 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:24.145917    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"
Jul 17 18:08:24.185523 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:24.185471    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94a8903-6da1-4e3e-ba03-4c81a943eacd-maas-api-service-ca\") pod \"f94a8903-6da1-4e3e-ba03-4c81a943eacd\" (UID: \"f94a8903-6da1-4e3e-ba03-4c81a943eacd\") "
Jul 17 18:08:24.185850 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:24.185827    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f94a8903-6da1-4e3e-ba03-4c81a943eacd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f94a8903-6da1-4e3e-ba03-4c81a943eacd" (UID: "f94a8903-6da1-4e3e-ba03-4c81a943eacd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:08:24.286917 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:24.286815    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94a8903-6da1-4e3e-ba03-4c81a943eacd-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:08:25.012412 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:25.012376    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"
Jul 17 18:08:25.012597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:25.012412    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs" event={"ID":"f94a8903-6da1-4e3e-ba03-4c81a943eacd","Type":"ContainerDied","Data":"810d796cccd209a15fb242a7ca3f3330b4807807601b45dac53f8b3fa7564172"}
Jul 17 18:08:25.012597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:25.012482    2572 scope.go:117] "RemoveContainer" containerID="5e0e325df83760ca9370c9d463ac6827b297c2573f99b631caafdad865d147e1"
Jul 17 18:08:25.030131 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:25.030092    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"]
Jul 17 18:08:25.033982 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:25.033950    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-lsjqs"]
Jul 17 18:08:26.521437 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:26.521401    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" path="/var/lib/kubelet/pods/f94a8903-6da1-4e3e-ba03-4c81a943eacd/volumes"
Jul 17 18:08:39.847427 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847387    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"]
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847757    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847770    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847779    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847785    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847796    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0434e8e3-7076-480f-8434-51757f6991ed" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847802    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0434e8e3-7076-480f-8434-51757f6991ed" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847808    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847814    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847829    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847835    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847845    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847850    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847858    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0434e8e3-7076-480f-8434-51757f6991ed" containerName="revoke-keys"
Jul 17 18:08:39.847897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847863    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0434e8e3-7076-480f-8434-51757f6991ed" containerName="revoke-keys"
Jul 17 18:08:39.848337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847918    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerName="revoke-keys"
Jul 17 18:08:39.848337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847924    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" containerName="revoke-keys"
Jul 17 18:08:39.848337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847931    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0434e8e3-7076-480f-8434-51757f6991ed" containerName="revoke-keys"
Jul 17 18:08:39.848337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847936    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0434e8e3-7076-480f-8434-51757f6991ed" containerName="revoke-keys"
Jul 17 18:08:39.848337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847942    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerName="revoke-keys"
Jul 17 18:08:39.848337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.847948    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" containerName="revoke-keys"
Jul 17 18:08:39.851098 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.851078    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"
Jul 17 18:08:39.854178 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.854142    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:08:39.854178 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.854159    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:08:39.860718 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.860689    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"]
Jul 17 18:08:39.926034 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:39.925976    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9b90e1-019e-48e7-9dba-bbbcdefd5384-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v\" (UID: \"1a9b90e1-019e-48e7-9dba-bbbcdefd5384\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"
Jul 17 18:08:40.026607 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:40.026559    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9b90e1-019e-48e7-9dba-bbbcdefd5384-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v\" (UID: \"1a9b90e1-019e-48e7-9dba-bbbcdefd5384\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"
Jul 17 18:08:40.027285 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:40.027264    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9b90e1-019e-48e7-9dba-bbbcdefd5384-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v\" (UID: \"1a9b90e1-019e-48e7-9dba-bbbcdefd5384\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"
Jul 17 18:08:40.162804 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:40.162711    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"
Jul 17 18:08:40.307580 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:40.307543    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"]
Jul 17 18:08:40.309406 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:08:40.309367    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9b90e1_019e_48e7_9dba_bbbcdefd5384.slice/crio-7816ac0fa6e1885ba8af3155f64a79c5be43da95a8d833415e3aca2a597232bf WatchSource:0}: Error finding container 7816ac0fa6e1885ba8af3155f64a79c5be43da95a8d833415e3aca2a597232bf: Status 404 returned error can't find the container with id 7816ac0fa6e1885ba8af3155f64a79c5be43da95a8d833415e3aca2a597232bf
Jul 17 18:08:41.101304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:41.101267    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" event={"ID":"1a9b90e1-019e-48e7-9dba-bbbcdefd5384","Type":"ContainerStarted","Data":"616f9e09a8e5a80aab0e3c2c56c6b719f76027aa910631b6ab33f4f38ae34541"}
Jul 17 18:08:41.101749 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:41.101324    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" event={"ID":"1a9b90e1-019e-48e7-9dba-bbbcdefd5384","Type":"ContainerStarted","Data":"7816ac0fa6e1885ba8af3155f64a79c5be43da95a8d833415e3aca2a597232bf"}
Jul 17 18:08:41.119232 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:41.119172    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" podStartSLOduration=2.119157579 podStartE2EDuration="2.119157579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:08:41.117664106 +0000 UTC m=+2693.123019973" watchObservedRunningTime="2026-07-17 18:08:41.119157579 +0000 UTC m=+2693.124513435"
Jul 17 18:08:42.108018 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:42.107981    2572 generic.go:356] "Generic (PLEG): container finished" podID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerID="616f9e09a8e5a80aab0e3c2c56c6b719f76027aa910631b6ab33f4f38ae34541" exitCode=6
Jul 17 18:08:42.108465 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:42.108069    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" event={"ID":"1a9b90e1-019e-48e7-9dba-bbbcdefd5384","Type":"ContainerDied","Data":"616f9e09a8e5a80aab0e3c2c56c6b719f76027aa910631b6ab33f4f38ae34541"}
Jul 17 18:08:42.108465 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:42.108229    2572 scope.go:117] "RemoveContainer" containerID="616f9e09a8e5a80aab0e3c2c56c6b719f76027aa910631b6ab33f4f38ae34541"
Jul 17 18:08:43.115148 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:43.115115    2572 generic.go:356] "Generic (PLEG): container finished" podID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerID="a0afb1612a3298a47180aed2fe655d47cd014a38623be0e9e61d76f919e53353" exitCode=6
Jul 17 18:08:43.115668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:43.115204    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" event={"ID":"1a9b90e1-019e-48e7-9dba-bbbcdefd5384","Type":"ContainerDied","Data":"a0afb1612a3298a47180aed2fe655d47cd014a38623be0e9e61d76f919e53353"}
Jul 17 18:08:43.115668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:43.115297    2572 scope.go:117] "RemoveContainer" containerID="616f9e09a8e5a80aab0e3c2c56c6b719f76027aa910631b6ab33f4f38ae34541"
Jul 17 18:08:43.115668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:43.115395    2572 scope.go:117] "RemoveContainer" containerID="a0afb1612a3298a47180aed2fe655d47cd014a38623be0e9e61d76f919e53353"
Jul 17 18:08:43.115668 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:43.115646    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v_odh-ai-gateway-infra(1a9b90e1-019e-48e7-9dba-bbbcdefd5384)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384"
Jul 17 18:08:44.122683 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.122654    2572 scope.go:117] "RemoveContainer" containerID="a0afb1612a3298a47180aed2fe655d47cd014a38623be0e9e61d76f919e53353"
Jul 17 18:08:44.123094 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:44.122842    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v_odh-ai-gateway-infra(1a9b90e1-019e-48e7-9dba-bbbcdefd5384)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384"
Jul 17 18:08:44.269836 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.269798    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"]
Jul 17 18:08:44.270145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.270133    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerName="revoke-keys"
Jul 17 18:08:44.270186 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.270147    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerName="revoke-keys"
Jul 17 18:08:44.270186 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.270162    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0434e8e3-7076-480f-8434-51757f6991ed" containerName="revoke-keys"
Jul 17 18:08:44.270186 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.270168    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0434e8e3-7076-480f-8434-51757f6991ed" containerName="revoke-keys"
Jul 17 18:08:44.270287 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.270234    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0434e8e3-7076-480f-8434-51757f6991ed" containerName="revoke-keys"
Jul 17 18:08:44.270287 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.270246    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcfb2453-d382-4a30-831e-8552837c59bf" containerName="revoke-keys"
Jul 17 18:08:44.270287 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.270253    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f94a8903-6da1-4e3e-ba03-4c81a943eacd" containerName="revoke-keys"
Jul 17 18:08:44.274431 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.274402    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"
Jul 17 18:08:44.282535 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.282040    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"]
Jul 17 18:08:44.368881 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.368837    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb95e5d-f84d-40e5-a94a-12bf80bddf10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d\" (UID: \"5bb95e5d-f84d-40e5-a94a-12bf80bddf10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"
Jul 17 18:08:44.470131 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.470055    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb95e5d-f84d-40e5-a94a-12bf80bddf10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d\" (UID: \"5bb95e5d-f84d-40e5-a94a-12bf80bddf10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"
Jul 17 18:08:44.470771 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.470747    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb95e5d-f84d-40e5-a94a-12bf80bddf10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d\" (UID: \"5bb95e5d-f84d-40e5-a94a-12bf80bddf10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"
Jul 17 18:08:44.587019 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.586978    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"
Jul 17 18:08:44.733283 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:44.733252    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"]
Jul 17 18:08:44.735053 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:08:44.735023    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb95e5d_f84d_40e5_a94a_12bf80bddf10.slice/crio-f115a30aafce9ad61940920cec9289c5368150b84033b96119fbf491d4926a55 WatchSource:0}: Error finding container f115a30aafce9ad61940920cec9289c5368150b84033b96119fbf491d4926a55: Status 404 returned error can't find the container with id f115a30aafce9ad61940920cec9289c5368150b84033b96119fbf491d4926a55
Jul 17 18:08:45.129675 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:45.129633    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" event={"ID":"5bb95e5d-f84d-40e5-a94a-12bf80bddf10","Type":"ContainerStarted","Data":"042fe901dc9ee4a61208e27127fe91f76eb315fccb4263dbb8784cd18feea4cf"}
Jul 17 18:08:45.130175 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:45.129706    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" event={"ID":"5bb95e5d-f84d-40e5-a94a-12bf80bddf10","Type":"ContainerStarted","Data":"f115a30aafce9ad61940920cec9289c5368150b84033b96119fbf491d4926a55"}
Jul 17 18:08:45.146044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:45.145982    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" podStartSLOduration=1.145965412 podStartE2EDuration="1.145965412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:08:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:08:45.145081473 +0000 UTC m=+2697.150437339" watchObservedRunningTime="2026-07-17 18:08:45.145965412 +0000 UTC m=+2697.151321267"
Jul 17 18:08:46.136821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:46.136787    2572 generic.go:356] "Generic (PLEG): container finished" podID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerID="042fe901dc9ee4a61208e27127fe91f76eb315fccb4263dbb8784cd18feea4cf" exitCode=6
Jul 17 18:08:46.137221 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:46.136868    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" event={"ID":"5bb95e5d-f84d-40e5-a94a-12bf80bddf10","Type":"ContainerDied","Data":"042fe901dc9ee4a61208e27127fe91f76eb315fccb4263dbb8784cd18feea4cf"}
Jul 17 18:08:46.137221 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:46.137006    2572 scope.go:117] "RemoveContainer" containerID="042fe901dc9ee4a61208e27127fe91f76eb315fccb4263dbb8784cd18feea4cf"
Jul 17 18:08:47.144361 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:47.144263    2572 generic.go:356] "Generic (PLEG): container finished" podID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerID="5b7f814b032f2512911e9c34551f37ce0550415b4ca81c41de9b66c99988d388" exitCode=6
Jul 17 18:08:47.144788 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:47.144353    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" event={"ID":"5bb95e5d-f84d-40e5-a94a-12bf80bddf10","Type":"ContainerDied","Data":"5b7f814b032f2512911e9c34551f37ce0550415b4ca81c41de9b66c99988d388"}
Jul 17 18:08:47.144788 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:47.144416    2572 scope.go:117] "RemoveContainer" containerID="042fe901dc9ee4a61208e27127fe91f76eb315fccb4263dbb8784cd18feea4cf"
Jul 17 18:08:47.144788 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:47.144658    2572 scope.go:117] "RemoveContainer" containerID="5b7f814b032f2512911e9c34551f37ce0550415b4ca81c41de9b66c99988d388"
Jul 17 18:08:47.144920 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:47.144890    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d_odh-ai-gateway-infra(5bb95e5d-f84d-40e5-a94a-12bf80bddf10)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10"
Jul 17 18:08:48.152260 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:48.152227    2572 scope.go:117] "RemoveContainer" containerID="5b7f814b032f2512911e9c34551f37ce0550415b4ca81c41de9b66c99988d388"
Jul 17 18:08:48.152722 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:48.152483    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d_odh-ai-gateway-infra(5bb95e5d-f84d-40e5-a94a-12bf80bddf10)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10"
Jul 17 18:08:48.693271 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:48.693244    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 18:08:48.700118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:48.700082    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 18:08:48.700379 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:48.700361    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 18:08:48.706382 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:48.706356    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 18:08:55.665705 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:55.665665    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"]
Jul 17 18:08:55.670999 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:55.670971    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"
Jul 17 18:08:55.682991 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:55.682953    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"]
Jul 17 18:08:55.768480 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:55.768441    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/710c0b14-159f-49ba-bca9-a86f3874c70a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p\" (UID: \"710c0b14-159f-49ba-bca9-a86f3874c70a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"
Jul 17 18:08:55.870118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:55.870074    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/710c0b14-159f-49ba-bca9-a86f3874c70a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p\" (UID: \"710c0b14-159f-49ba-bca9-a86f3874c70a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"
Jul 17 18:08:55.870795 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:55.870772    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/710c0b14-159f-49ba-bca9-a86f3874c70a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p\" (UID: \"710c0b14-159f-49ba-bca9-a86f3874c70a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"
Jul 17 18:08:55.982546 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:55.982388    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"
Jul 17 18:08:56.128783 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:56.128754    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"]
Jul 17 18:08:56.131672 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:08:56.131637    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod710c0b14_159f_49ba_bca9_a86f3874c70a.slice/crio-22b6fe9799e0c4cc5bbcb849e6f3d363ce3fa3d13d3584879f338aac2407a7b6 WatchSource:0}: Error finding container 22b6fe9799e0c4cc5bbcb849e6f3d363ce3fa3d13d3584879f338aac2407a7b6: Status 404 returned error can't find the container with id 22b6fe9799e0c4cc5bbcb849e6f3d363ce3fa3d13d3584879f338aac2407a7b6
Jul 17 18:08:56.196056 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:56.196016    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p" event={"ID":"710c0b14-159f-49ba-bca9-a86f3874c70a","Type":"ContainerStarted","Data":"22b6fe9799e0c4cc5bbcb849e6f3d363ce3fa3d13d3584879f338aac2407a7b6"}
Jul 17 18:08:56.580222 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:56.580181    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"]
Jul 17 18:08:56.584068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:56.584047    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"
Jul 17 18:08:56.592422 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:56.592390    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"]
Jul 17 18:08:56.680105 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:56.680060    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b558f9-6ea4-43b9-83da-002e5282ad44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l\" (UID: \"30b558f9-6ea4-43b9-83da-002e5282ad44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"
Jul 17 18:08:56.781501 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:56.781447    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b558f9-6ea4-43b9-83da-002e5282ad44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l\" (UID: \"30b558f9-6ea4-43b9-83da-002e5282ad44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"
Jul 17 18:08:56.782139 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:56.782112    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b558f9-6ea4-43b9-83da-002e5282ad44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l\" (UID: \"30b558f9-6ea4-43b9-83da-002e5282ad44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"
Jul 17 18:08:56.896981 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:56.896867    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"
Jul 17 18:08:57.039115 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:57.039074    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"]
Jul 17 18:08:57.040287 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:08:57.040251    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b558f9_6ea4_43b9_83da_002e5282ad44.slice/crio-c34ea1074e14a48a301b086b456a9045fb9e0b4d6ca56565b5fe5058c3cfc02f WatchSource:0}: Error finding container c34ea1074e14a48a301b086b456a9045fb9e0b4d6ca56565b5fe5058c3cfc02f: Status 404 returned error can't find the container with id c34ea1074e14a48a301b086b456a9045fb9e0b4d6ca56565b5fe5058c3cfc02f
Jul 17 18:08:57.203127 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:57.203019    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l" event={"ID":"30b558f9-6ea4-43b9-83da-002e5282ad44","Type":"ContainerStarted","Data":"c34ea1074e14a48a301b086b456a9045fb9e0b4d6ca56565b5fe5058c3cfc02f"}
Jul 17 18:08:57.204593 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:57.204562    2572 generic.go:356] "Generic (PLEG): container finished" podID="710c0b14-159f-49ba-bca9-a86f3874c70a" containerID="903933f09b5aa692693fd45a5d045828745fe8b04da9152229f5656084e6908a" exitCode=0
Jul 17 18:08:57.204752 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:57.204649    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p" event={"ID":"710c0b14-159f-49ba-bca9-a86f3874c70a","Type":"ContainerDied","Data":"903933f09b5aa692693fd45a5d045828745fe8b04da9152229f5656084e6908a"}
Jul 17 18:08:58.212241 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.212201    2572 generic.go:356] "Generic (PLEG): container finished" podID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerID="e17524af4282c7df559d474a05f5021c27b3889b401afed267746ea3f80efbea" exitCode=6
Jul 17 18:08:58.212701 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.212289    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l" event={"ID":"30b558f9-6ea4-43b9-83da-002e5282ad44","Type":"ContainerDied","Data":"e17524af4282c7df559d474a05f5021c27b3889b401afed267746ea3f80efbea"}
Jul 17 18:08:58.212701 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.212480    2572 scope.go:117] "RemoveContainer" containerID="e17524af4282c7df559d474a05f5021c27b3889b401afed267746ea3f80efbea"
Jul 17 18:08:58.366380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.366349    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"
Jul 17 18:08:58.427808 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.427778    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"]
Jul 17 18:08:58.428144 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.428133    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="710c0b14-159f-49ba-bca9-a86f3874c70a" containerName="revoke-keys"
Jul 17 18:08:58.428191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.428145    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="710c0b14-159f-49ba-bca9-a86f3874c70a" containerName="revoke-keys"
Jul 17 18:08:58.428232 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.428220    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="710c0b14-159f-49ba-bca9-a86f3874c70a" containerName="revoke-keys"
Jul 17 18:08:58.431217 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.431200    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"
Jul 17 18:08:58.440013 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.439980    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"]
Jul 17 18:08:58.499615 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.499516    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/710c0b14-159f-49ba-bca9-a86f3874c70a-maas-api-service-ca\") pod \"710c0b14-159f-49ba-bca9-a86f3874c70a\" (UID: \"710c0b14-159f-49ba-bca9-a86f3874c70a\") "
Jul 17 18:08:58.499894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.499869    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/710c0b14-159f-49ba-bca9-a86f3874c70a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "710c0b14-159f-49ba-bca9-a86f3874c70a" (UID: "710c0b14-159f-49ba-bca9-a86f3874c70a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:08:58.512910 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.512880    2572 scope.go:117] "RemoveContainer" containerID="a0afb1612a3298a47180aed2fe655d47cd014a38623be0e9e61d76f919e53353"
Jul 17 18:08:58.600689 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.600639    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/804b460b-afe6-489b-9e6f-ab44c1d59720-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6\" (UID: \"804b460b-afe6-489b-9e6f-ab44c1d59720\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"
Jul 17 18:08:58.600897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.600820    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/710c0b14-159f-49ba-bca9-a86f3874c70a-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:08:58.702159 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.702115    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/804b460b-afe6-489b-9e6f-ab44c1d59720-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6\" (UID: \"804b460b-afe6-489b-9e6f-ab44c1d59720\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"
Jul 17 18:08:58.702855 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.702834    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/804b460b-afe6-489b-9e6f-ab44c1d59720-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6\" (UID: \"804b460b-afe6-489b-9e6f-ab44c1d59720\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"
Jul 17 18:08:58.760743 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.760632    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"
Jul 17 18:08:58.915162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:58.915133    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"]
Jul 17 18:08:58.916379 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:08:58.916351    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod804b460b_afe6_489b_9e6f_ab44c1d59720.slice/crio-89670c6dfff003a50598f01fc104d81cf0f10b2326fc13c7f0fbc8a870fbb3f7 WatchSource:0}: Error finding container 89670c6dfff003a50598f01fc104d81cf0f10b2326fc13c7f0fbc8a870fbb3f7: Status 404 returned error can't find the container with id 89670c6dfff003a50598f01fc104d81cf0f10b2326fc13c7f0fbc8a870fbb3f7
Jul 17 18:08:59.219601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:59.219566    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p"
Jul 17 18:08:59.219601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:59.219572    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p" event={"ID":"710c0b14-159f-49ba-bca9-a86f3874c70a","Type":"ContainerDied","Data":"22b6fe9799e0c4cc5bbcb849e6f3d363ce3fa3d13d3584879f338aac2407a7b6"}
Jul 17 18:08:59.220090 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:59.219627    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22b6fe9799e0c4cc5bbcb849e6f3d363ce3fa3d13d3584879f338aac2407a7b6"
Jul 17 18:08:59.222100 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:59.221945    2572 generic.go:356] "Generic (PLEG): container finished" podID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerID="75f6f2b214b658ff38984c263d070ad85c3abc27b304a54f0bc499f0bd332412" exitCode=6
Jul 17 18:08:59.222100 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:59.222009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l" event={"ID":"30b558f9-6ea4-43b9-83da-002e5282ad44","Type":"ContainerDied","Data":"75f6f2b214b658ff38984c263d070ad85c3abc27b304a54f0bc499f0bd332412"}
Jul 17 18:08:59.222100 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:59.222091    2572 scope.go:117] "RemoveContainer" containerID="e17524af4282c7df559d474a05f5021c27b3889b401afed267746ea3f80efbea"
Jul 17 18:08:59.222396 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:59.222289    2572 scope.go:117] "RemoveContainer" containerID="75f6f2b214b658ff38984c263d070ad85c3abc27b304a54f0bc499f0bd332412"
Jul 17 18:08:59.222835 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:08:59.222583    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l_odh-ai-gateway-infra(30b558f9-6ea4-43b9-83da-002e5282ad44)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44"
Jul 17 18:08:59.225087 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:59.225061    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" event={"ID":"1a9b90e1-019e-48e7-9dba-bbbcdefd5384","Type":"ContainerStarted","Data":"1d56fed62feb59789912bd0d209f017d6ffa83c0a50ccc37a0cfd2043de1c20b"}
Jul 17 18:08:59.226929 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:08:59.226905    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6" event={"ID":"804b460b-afe6-489b-9e6f-ab44c1d59720","Type":"ContainerStarted","Data":"89670c6dfff003a50598f01fc104d81cf0f10b2326fc13c7f0fbc8a870fbb3f7"}
Jul 17 18:09:00.235284 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.235251    2572 scope.go:117] "RemoveContainer" containerID="75f6f2b214b658ff38984c263d070ad85c3abc27b304a54f0bc499f0bd332412"
Jul 17 18:09:00.235793 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:00.235515    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l_odh-ai-gateway-infra(30b558f9-6ea4-43b9-83da-002e5282ad44)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44"
Jul 17 18:09:00.237024 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.236998    2572 generic.go:356] "Generic (PLEG): container finished" podID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerID="1d56fed62feb59789912bd0d209f017d6ffa83c0a50ccc37a0cfd2043de1c20b" exitCode=6
Jul 17 18:09:00.237173 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.237072    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" event={"ID":"1a9b90e1-019e-48e7-9dba-bbbcdefd5384","Type":"ContainerDied","Data":"1d56fed62feb59789912bd0d209f017d6ffa83c0a50ccc37a0cfd2043de1c20b"}
Jul 17 18:09:00.237173 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.237136    2572 scope.go:117] "RemoveContainer" containerID="a0afb1612a3298a47180aed2fe655d47cd014a38623be0e9e61d76f919e53353"
Jul 17 18:09:00.237477 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.237455    2572 scope.go:117] "RemoveContainer" containerID="1d56fed62feb59789912bd0d209f017d6ffa83c0a50ccc37a0cfd2043de1c20b"
Jul 17 18:09:00.237722 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:00.237692    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v_odh-ai-gateway-infra(1a9b90e1-019e-48e7-9dba-bbbcdefd5384)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384"
Jul 17 18:09:00.238744 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.238718    2572 generic.go:356] "Generic (PLEG): container finished" podID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerID="2545906e0046704590ca1e2ad0fb8e4da1177a7ba06874f04ac1a63c0631dd0b" exitCode=6
Jul 17 18:09:00.238843 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.238785    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6" event={"ID":"804b460b-afe6-489b-9e6f-ab44c1d59720","Type":"ContainerDied","Data":"2545906e0046704590ca1e2ad0fb8e4da1177a7ba06874f04ac1a63c0631dd0b"}
Jul 17 18:09:00.238964 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.238950    2572 scope.go:117] "RemoveContainer" containerID="2545906e0046704590ca1e2ad0fb8e4da1177a7ba06874f04ac1a63c0631dd0b"
Jul 17 18:09:00.269094 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.269062    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"]
Jul 17 18:09:00.516354 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:00.516274    2572 scope.go:117] "RemoveContainer" containerID="5b7f814b032f2512911e9c34551f37ce0550415b4ca81c41de9b66c99988d388"
Jul 17 18:09:01.245979 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.245949    2572 generic.go:356] "Generic (PLEG): container finished" podID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerID="d9b45fdffd1f468174eccaff678bacecc55be3b5c185624d5cc77a7d06fbfdbd" exitCode=6
Jul 17 18:09:01.246444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.246027    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" event={"ID":"5bb95e5d-f84d-40e5-a94a-12bf80bddf10","Type":"ContainerDied","Data":"d9b45fdffd1f468174eccaff678bacecc55be3b5c185624d5cc77a7d06fbfdbd"}
Jul 17 18:09:01.246444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.246098    2572 scope.go:117] "RemoveContainer" containerID="5b7f814b032f2512911e9c34551f37ce0550415b4ca81c41de9b66c99988d388"
Jul 17 18:09:01.246444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.246412    2572 scope.go:117] "RemoveContainer" containerID="d9b45fdffd1f468174eccaff678bacecc55be3b5c185624d5cc77a7d06fbfdbd"
Jul 17 18:09:01.246717 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:01.246697    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d_odh-ai-gateway-infra(5bb95e5d-f84d-40e5-a94a-12bf80bddf10)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10"
Jul 17 18:09:01.249752 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.249730    2572 generic.go:356] "Generic (PLEG): container finished" podID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerID="246b30d0da4f77453e2cec10330f638e28124a90335d16ad681120dab2eedb17" exitCode=6
Jul 17 18:09:01.249850 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.249761    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6" event={"ID":"804b460b-afe6-489b-9e6f-ab44c1d59720","Type":"ContainerDied","Data":"246b30d0da4f77453e2cec10330f638e28124a90335d16ad681120dab2eedb17"}
Jul 17 18:09:01.250048 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.250033    2572 scope.go:117] "RemoveContainer" containerID="246b30d0da4f77453e2cec10330f638e28124a90335d16ad681120dab2eedb17"
Jul 17 18:09:01.250216 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:01.250199    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6_odh-ai-gateway-infra(804b460b-afe6-489b-9e6f-ab44c1d59720)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720"
Jul 17 18:09:01.274516 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.274468    2572 scope.go:117] "RemoveContainer" containerID="2545906e0046704590ca1e2ad0fb8e4da1177a7ba06874f04ac1a63c0631dd0b"
Jul 17 18:09:01.399868 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.399836    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"
Jul 17 18:09:01.532311 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.532212    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9b90e1-019e-48e7-9dba-bbbcdefd5384-maas-api-service-ca\") pod \"1a9b90e1-019e-48e7-9dba-bbbcdefd5384\" (UID: \"1a9b90e1-019e-48e7-9dba-bbbcdefd5384\") "
Jul 17 18:09:01.532643 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.532621    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a9b90e1-019e-48e7-9dba-bbbcdefd5384-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a9b90e1-019e-48e7-9dba-bbbcdefd5384" (UID: "1a9b90e1-019e-48e7-9dba-bbbcdefd5384"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:09:01.633665 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:01.633624    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9b90e1-019e-48e7-9dba-bbbcdefd5384-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:09:02.260563 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.260523    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v" event={"ID":"1a9b90e1-019e-48e7-9dba-bbbcdefd5384","Type":"ContainerDied","Data":"7816ac0fa6e1885ba8af3155f64a79c5be43da95a8d833415e3aca2a597232bf"}
Jul 17 18:09:02.261074 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.260533    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"
Jul 17 18:09:02.261074 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.260597    2572 scope.go:117] "RemoveContainer" containerID="1d56fed62feb59789912bd0d209f017d6ffa83c0a50ccc37a0cfd2043de1c20b"
Jul 17 18:09:02.262672 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.262651    2572 scope.go:117] "RemoveContainer" containerID="246b30d0da4f77453e2cec10330f638e28124a90335d16ad681120dab2eedb17"
Jul 17 18:09:02.262849 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:02.262827    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6_odh-ai-gateway-infra(804b460b-afe6-489b-9e6f-ab44c1d59720)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720"
Jul 17 18:09:02.272459 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.272430    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"]
Jul 17 18:09:02.298953 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.298906    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"]
Jul 17 18:09:02.305310 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.305275    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xrw9v"]
Jul 17 18:09:02.417242 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.417215    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"
Jul 17 18:09:02.516857 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.516772    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" path="/var/lib/kubelet/pods/1a9b90e1-019e-48e7-9dba-bbbcdefd5384/volumes"
Jul 17 18:09:02.542113 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.542079    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb95e5d-f84d-40e5-a94a-12bf80bddf10-maas-api-service-ca\") pod \"5bb95e5d-f84d-40e5-a94a-12bf80bddf10\" (UID: \"5bb95e5d-f84d-40e5-a94a-12bf80bddf10\") "
Jul 17 18:09:02.542477 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.542455    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bb95e5d-f84d-40e5-a94a-12bf80bddf10-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5bb95e5d-f84d-40e5-a94a-12bf80bddf10" (UID: "5bb95e5d-f84d-40e5-a94a-12bf80bddf10"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:09:02.643442 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:02.643398    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb95e5d-f84d-40e5-a94a-12bf80bddf10-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:09:03.270279 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:03.270248    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"
Jul 17 18:09:03.270784 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:03.270283    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d" event={"ID":"5bb95e5d-f84d-40e5-a94a-12bf80bddf10","Type":"ContainerDied","Data":"f115a30aafce9ad61940920cec9289c5368150b84033b96119fbf491d4926a55"}
Jul 17 18:09:03.270784 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:03.270349    2572 scope.go:117] "RemoveContainer" containerID="d9b45fdffd1f468174eccaff678bacecc55be3b5c185624d5cc77a7d06fbfdbd"
Jul 17 18:09:03.294924 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:03.294892    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"]
Jul 17 18:09:03.302845 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:03.302811    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bj54d"]
Jul 17 18:09:04.515844 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:04.515810    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" path="/var/lib/kubelet/pods/5bb95e5d-f84d-40e5-a94a-12bf80bddf10/volumes"
Jul 17 18:09:14.510437 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:14.510388    2572 scope.go:117] "RemoveContainer" containerID="75f6f2b214b658ff38984c263d070ad85c3abc27b304a54f0bc499f0bd332412"
Jul 17 18:09:15.347064 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:15.347033    2572 generic.go:356] "Generic (PLEG): container finished" podID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerID="a244a36caef4e5b612dd0fc975e5e4a1af62dcd24bac0029b90ac1f0ebf13349" exitCode=6
Jul 17 18:09:15.347253 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:15.347078    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l" event={"ID":"30b558f9-6ea4-43b9-83da-002e5282ad44","Type":"ContainerDied","Data":"a244a36caef4e5b612dd0fc975e5e4a1af62dcd24bac0029b90ac1f0ebf13349"}
Jul 17 18:09:15.347253 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:15.347130    2572 scope.go:117] "RemoveContainer" containerID="75f6f2b214b658ff38984c263d070ad85c3abc27b304a54f0bc499f0bd332412"
Jul 17 18:09:15.347466 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:15.347435    2572 scope.go:117] "RemoveContainer" containerID="a244a36caef4e5b612dd0fc975e5e4a1af62dcd24bac0029b90ac1f0ebf13349"
Jul 17 18:09:15.347692 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:15.347672    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l_odh-ai-gateway-infra(30b558f9-6ea4-43b9-83da-002e5282ad44)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44"
Jul 17 18:09:16.377054 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:16.377014    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"]
Jul 17 18:09:16.515288 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:16.515261    2572 scope.go:117] "RemoveContainer" containerID="246b30d0da4f77453e2cec10330f638e28124a90335d16ad681120dab2eedb17"
Jul 17 18:09:16.515858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:16.515696    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"
Jul 17 18:09:16.663104 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:16.663006    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b558f9-6ea4-43b9-83da-002e5282ad44-maas-api-service-ca\") pod \"30b558f9-6ea4-43b9-83da-002e5282ad44\" (UID: \"30b558f9-6ea4-43b9-83da-002e5282ad44\") "
Jul 17 18:09:16.663385 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:16.663362    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30b558f9-6ea4-43b9-83da-002e5282ad44-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "30b558f9-6ea4-43b9-83da-002e5282ad44" (UID: "30b558f9-6ea4-43b9-83da-002e5282ad44"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:09:16.763753 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:16.763711    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30b558f9-6ea4-43b9-83da-002e5282ad44-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:09:17.360138 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:17.360102    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l" event={"ID":"30b558f9-6ea4-43b9-83da-002e5282ad44","Type":"ContainerDied","Data":"c34ea1074e14a48a301b086b456a9045fb9e0b4d6ca56565b5fe5058c3cfc02f"}
Jul 17 18:09:17.360361 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:17.360117    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"
Jul 17 18:09:17.360361 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:17.360169    2572 scope.go:117] "RemoveContainer" containerID="a244a36caef4e5b612dd0fc975e5e4a1af62dcd24bac0029b90ac1f0ebf13349"
Jul 17 18:09:17.362246 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:17.362219    2572 generic.go:356] "Generic (PLEG): container finished" podID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerID="3381519b73276a0aa6a688d1753a7b3ae26cbb4a74b1634b06d6064b42d08bcb" exitCode=6
Jul 17 18:09:17.362359 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:17.362286    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6" event={"ID":"804b460b-afe6-489b-9e6f-ab44c1d59720","Type":"ContainerDied","Data":"3381519b73276a0aa6a688d1753a7b3ae26cbb4a74b1634b06d6064b42d08bcb"}
Jul 17 18:09:17.362643 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:17.362626    2572 scope.go:117] "RemoveContainer" containerID="3381519b73276a0aa6a688d1753a7b3ae26cbb4a74b1634b06d6064b42d08bcb"
Jul 17 18:09:17.362848 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:17.362830    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6_odh-ai-gateway-infra(804b460b-afe6-489b-9e6f-ab44c1d59720)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720"
Jul 17 18:09:17.369899 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:17.369876    2572 scope.go:117] "RemoveContainer" containerID="246b30d0da4f77453e2cec10330f638e28124a90335d16ad681120dab2eedb17"
Jul 17 18:09:17.398611 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:17.398528    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"]
Jul 17 18:09:17.401461 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:17.401433    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-vnn7l"]
Jul 17 18:09:18.389264 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:18.389225    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"]
Jul 17 18:09:18.515398 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:18.515367    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" path="/var/lib/kubelet/pods/30b558f9-6ea4-43b9-83da-002e5282ad44/volumes"
Jul 17 18:09:18.540821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:18.540790    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"
Jul 17 18:09:18.582935 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:18.582897    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/804b460b-afe6-489b-9e6f-ab44c1d59720-maas-api-service-ca\") pod \"804b460b-afe6-489b-9e6f-ab44c1d59720\" (UID: \"804b460b-afe6-489b-9e6f-ab44c1d59720\") "
Jul 17 18:09:18.583299 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:18.583269    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/804b460b-afe6-489b-9e6f-ab44c1d59720-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "804b460b-afe6-489b-9e6f-ab44c1d59720" (UID: "804b460b-afe6-489b-9e6f-ab44c1d59720"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:09:18.683899 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:18.683795    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/804b460b-afe6-489b-9e6f-ab44c1d59720-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:09:19.377045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:19.377016    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"
Jul 17 18:09:19.377045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:19.377035    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6" event={"ID":"804b460b-afe6-489b-9e6f-ab44c1d59720","Type":"ContainerDied","Data":"89670c6dfff003a50598f01fc104d81cf0f10b2326fc13c7f0fbc8a870fbb3f7"}
Jul 17 18:09:19.377295 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:19.377096    2572 scope.go:117] "RemoveContainer" containerID="3381519b73276a0aa6a688d1753a7b3ae26cbb4a74b1634b06d6064b42d08bcb"
Jul 17 18:09:19.401967 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:19.401930    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"]
Jul 17 18:09:19.411907 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:19.411871    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-9kfh6"]
Jul 17 18:09:20.515995 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:20.515953    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" path="/var/lib/kubelet/pods/804b460b-afe6-489b-9e6f-ab44c1d59720/volumes"
Jul 17 18:09:34.314361 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314317    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"]
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314681    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314693    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314702    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314708    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314715    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314723    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314733    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314738    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314745    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314750    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314756    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314762    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314771    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314776    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314781    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314786    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314794    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314799    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314804    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerName="revoke-keys"
Jul 17 18:09:34.314824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314810    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerName="revoke-keys"
Jul 17 18:09:34.315446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314880    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerName="revoke-keys"
Jul 17 18:09:34.315446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314887    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerName="revoke-keys"
Jul 17 18:09:34.315446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314894    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerName="revoke-keys"
Jul 17 18:09:34.315446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314899    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerName="revoke-keys"
Jul 17 18:09:34.315446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314905    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerName="revoke-keys"
Jul 17 18:09:34.315446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314911    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerName="revoke-keys"
Jul 17 18:09:34.315446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314917    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerName="revoke-keys"
Jul 17 18:09:34.315446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314922    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bb95e5d-f84d-40e5-a94a-12bf80bddf10" containerName="revoke-keys"
Jul 17 18:09:34.315446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.314929    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerName="revoke-keys"
Jul 17 18:09:34.318023 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.318002    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"
Jul 17 18:09:34.320649 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.320625    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:09:34.320649 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.320643    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:09:34.325999 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.325972    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"]
Jul 17 18:09:34.417690 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.417650    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef08ee4-98b8-4428-a3da-2a1bef563a09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8\" (UID: \"3ef08ee4-98b8-4428-a3da-2a1bef563a09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"
Jul 17 18:09:34.518266 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.518231    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef08ee4-98b8-4428-a3da-2a1bef563a09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8\" (UID: \"3ef08ee4-98b8-4428-a3da-2a1bef563a09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"
Jul 17 18:09:34.518877 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.518857    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef08ee4-98b8-4428-a3da-2a1bef563a09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8\" (UID: \"3ef08ee4-98b8-4428-a3da-2a1bef563a09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"
Jul 17 18:09:34.629292 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.629198    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"
Jul 17 18:09:34.771421 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:34.771395    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"]
Jul 17 18:09:34.773338 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:09:34.773305    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef08ee4_98b8_4428_a3da_2a1bef563a09.slice/crio-8c541cf5579e4d66a420d0a3d208c6e11aade73c5c451f6a998332262e93b336 WatchSource:0}: Error finding container 8c541cf5579e4d66a420d0a3d208c6e11aade73c5c451f6a998332262e93b336: Status 404 returned error can't find the container with id 8c541cf5579e4d66a420d0a3d208c6e11aade73c5c451f6a998332262e93b336
Jul 17 18:09:35.474023 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:35.473976    2572 generic.go:356] "Generic (PLEG): container finished" podID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerID="f25b7e7b72159fc73b7cb97350ea899feb6cc20342deb659d3ed973d94b4ff89" exitCode=6
Jul 17 18:09:35.474625 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:35.474290    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8" event={"ID":"3ef08ee4-98b8-4428-a3da-2a1bef563a09","Type":"ContainerDied","Data":"f25b7e7b72159fc73b7cb97350ea899feb6cc20342deb659d3ed973d94b4ff89"}
Jul 17 18:09:35.474625 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:35.474388    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8" event={"ID":"3ef08ee4-98b8-4428-a3da-2a1bef563a09","Type":"ContainerStarted","Data":"8c541cf5579e4d66a420d0a3d208c6e11aade73c5c451f6a998332262e93b336"}
Jul 17 18:09:35.474625 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:35.474540    2572 scope.go:117] "RemoveContainer" containerID="f25b7e7b72159fc73b7cb97350ea899feb6cc20342deb659d3ed973d94b4ff89"
Jul 17 18:09:36.481564 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:36.481466    2572 generic.go:356] "Generic (PLEG): container finished" podID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerID="1e75e4103148bdd7500861e0a09703a8e657be2773a2668fb9bf4e651bf6f2a8" exitCode=6
Jul 17 18:09:36.481984 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:36.481563    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8" event={"ID":"3ef08ee4-98b8-4428-a3da-2a1bef563a09","Type":"ContainerDied","Data":"1e75e4103148bdd7500861e0a09703a8e657be2773a2668fb9bf4e651bf6f2a8"}
Jul 17 18:09:36.481984 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:36.481625    2572 scope.go:117] "RemoveContainer" containerID="f25b7e7b72159fc73b7cb97350ea899feb6cc20342deb659d3ed973d94b4ff89"
Jul 17 18:09:36.481984 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:36.481922    2572 scope.go:117] "RemoveContainer" containerID="1e75e4103148bdd7500861e0a09703a8e657be2773a2668fb9bf4e651bf6f2a8"
Jul 17 18:09:36.482166 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:36.482145    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8_odh-ai-gateway-infra(3ef08ee4-98b8-4428-a3da-2a1bef563a09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09"
Jul 17 18:09:37.488732 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:37.488704    2572 scope.go:117] "RemoveContainer" containerID="1e75e4103148bdd7500861e0a09703a8e657be2773a2668fb9bf4e651bf6f2a8"
Jul 17 18:09:37.489153 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:37.488917    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8_odh-ai-gateway-infra(3ef08ee4-98b8-4428-a3da-2a1bef563a09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09"
Jul 17 18:09:39.891824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.891786    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"]
Jul 17 18:09:39.892231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.892121    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerName="revoke-keys"
Jul 17 18:09:39.892231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.892133    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerName="revoke-keys"
Jul 17 18:09:39.892231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.892149    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerName="revoke-keys"
Jul 17 18:09:39.892231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.892156    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerName="revoke-keys"
Jul 17 18:09:39.892231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.892222    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="30b558f9-6ea4-43b9-83da-002e5282ad44" containerName="revoke-keys"
Jul 17 18:09:39.892231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.892228    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="804b460b-afe6-489b-9e6f-ab44c1d59720" containerName="revoke-keys"
Jul 17 18:09:39.892444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.892235    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a9b90e1-019e-48e7-9dba-bbbcdefd5384" containerName="revoke-keys"
Jul 17 18:09:39.896464 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.896443    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"
Jul 17 18:09:39.904741 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.904713    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"]
Jul 17 18:09:39.968355 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:39.968308    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c56a6a7e-2f5c-410a-8b76-1ab361c0af05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb\" (UID: \"c56a6a7e-2f5c-410a-8b76-1ab361c0af05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"
Jul 17 18:09:40.068846 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:40.068802    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c56a6a7e-2f5c-410a-8b76-1ab361c0af05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb\" (UID: \"c56a6a7e-2f5c-410a-8b76-1ab361c0af05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"
Jul 17 18:09:40.069462 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:40.069441    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c56a6a7e-2f5c-410a-8b76-1ab361c0af05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb\" (UID: \"c56a6a7e-2f5c-410a-8b76-1ab361c0af05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"
Jul 17 18:09:40.208980 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:40.208880    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"
Jul 17 18:09:40.353071 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:40.352857    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"]
Jul 17 18:09:40.506882 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:40.506788    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb" event={"ID":"c56a6a7e-2f5c-410a-8b76-1ab361c0af05","Type":"ContainerStarted","Data":"2372ad92c0651d94503aa4c6dfee93756866883e0728a7333a2faabe5379e594"}
Jul 17 18:09:41.514004 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:41.513972    2572 generic.go:356] "Generic (PLEG): container finished" podID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerID="0209697493b46133671c77701e67fec9da5745bd2fbec2cc0435c65d78a9c578" exitCode=6
Jul 17 18:09:41.514444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:41.514019    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb" event={"ID":"c56a6a7e-2f5c-410a-8b76-1ab361c0af05","Type":"ContainerDied","Data":"0209697493b46133671c77701e67fec9da5745bd2fbec2cc0435c65d78a9c578"}
Jul 17 18:09:41.514444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:41.514226    2572 scope.go:117] "RemoveContainer" containerID="0209697493b46133671c77701e67fec9da5745bd2fbec2cc0435c65d78a9c578"
Jul 17 18:09:42.522148 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:42.522056    2572 generic.go:356] "Generic (PLEG): container finished" podID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerID="017b6147317ec6091dc91aaeb605f5067bc55533d664e1adfde429ae6e368a03" exitCode=6
Jul 17 18:09:42.522626 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:42.522142    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb" event={"ID":"c56a6a7e-2f5c-410a-8b76-1ab361c0af05","Type":"ContainerDied","Data":"017b6147317ec6091dc91aaeb605f5067bc55533d664e1adfde429ae6e368a03"}
Jul 17 18:09:42.522626 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:42.522201    2572 scope.go:117] "RemoveContainer" containerID="0209697493b46133671c77701e67fec9da5745bd2fbec2cc0435c65d78a9c578"
Jul 17 18:09:42.522626 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:42.522341    2572 scope.go:117] "RemoveContainer" containerID="017b6147317ec6091dc91aaeb605f5067bc55533d664e1adfde429ae6e368a03"
Jul 17 18:09:42.522740 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:42.522640    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb_odh-ai-gateway-infra(c56a6a7e-2f5c-410a-8b76-1ab361c0af05)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05"
Jul 17 18:09:43.529833 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:43.529798    2572 scope.go:117] "RemoveContainer" containerID="017b6147317ec6091dc91aaeb605f5067bc55533d664e1adfde429ae6e368a03"
Jul 17 18:09:43.530254 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:43.530017    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb_odh-ai-gateway-infra(c56a6a7e-2f5c-410a-8b76-1ab361c0af05)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05"
Jul 17 18:09:51.510028 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:51.509991    2572 scope.go:117] "RemoveContainer" containerID="1e75e4103148bdd7500861e0a09703a8e657be2773a2668fb9bf4e651bf6f2a8"
Jul 17 18:09:52.578895 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:52.578841    2572 generic.go:356] "Generic (PLEG): container finished" podID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerID="b3ab12977344f2b9c4919a702faa26a697bd35ad3b96d996d507a591e7d88933" exitCode=6
Jul 17 18:09:52.579362 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:52.578933    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8" event={"ID":"3ef08ee4-98b8-4428-a3da-2a1bef563a09","Type":"ContainerDied","Data":"b3ab12977344f2b9c4919a702faa26a697bd35ad3b96d996d507a591e7d88933"}
Jul 17 18:09:52.579362 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:52.579017    2572 scope.go:117] "RemoveContainer" containerID="1e75e4103148bdd7500861e0a09703a8e657be2773a2668fb9bf4e651bf6f2a8"
Jul 17 18:09:52.579527 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:52.579378    2572 scope.go:117] "RemoveContainer" containerID="b3ab12977344f2b9c4919a702faa26a697bd35ad3b96d996d507a591e7d88933"
Jul 17 18:09:52.579681 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:52.579649    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8_odh-ai-gateway-infra(3ef08ee4-98b8-4428-a3da-2a1bef563a09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09"
Jul 17 18:09:53.608015 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:53.607967    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"]
Jul 17 18:09:53.753417 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:53.753386    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"
Jul 17 18:09:53.895825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:53.895722    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef08ee4-98b8-4428-a3da-2a1bef563a09-maas-api-service-ca\") pod \"3ef08ee4-98b8-4428-a3da-2a1bef563a09\" (UID: \"3ef08ee4-98b8-4428-a3da-2a1bef563a09\") "
Jul 17 18:09:53.896170 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:53.896147    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef08ee4-98b8-4428-a3da-2a1bef563a09-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3ef08ee4-98b8-4428-a3da-2a1bef563a09" (UID: "3ef08ee4-98b8-4428-a3da-2a1bef563a09"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:09:53.996470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:53.996428    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef08ee4-98b8-4428-a3da-2a1bef563a09-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:09:54.593719 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:54.593683    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8" event={"ID":"3ef08ee4-98b8-4428-a3da-2a1bef563a09","Type":"ContainerDied","Data":"8c541cf5579e4d66a420d0a3d208c6e11aade73c5c451f6a998332262e93b336"}
Jul 17 18:09:54.593940 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:54.593747    2572 scope.go:117] "RemoveContainer" containerID="b3ab12977344f2b9c4919a702faa26a697bd35ad3b96d996d507a591e7d88933"
Jul 17 18:09:54.593940 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:54.593696    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"
Jul 17 18:09:54.613362 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:54.613326    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"]
Jul 17 18:09:54.622352 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:54.622315    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-hsms8"]
Jul 17 18:09:56.516377 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.516342    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" path="/var/lib/kubelet/pods/3ef08ee4-98b8-4428-a3da-2a1bef563a09/volumes"
Jul 17 18:09:56.624641 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.624603    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"]
Jul 17 18:09:56.625003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.624990    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerName="revoke-keys"
Jul 17 18:09:56.625050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.625010    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerName="revoke-keys"
Jul 17 18:09:56.625050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.625030    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerName="revoke-keys"
Jul 17 18:09:56.625050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.625036    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerName="revoke-keys"
Jul 17 18:09:56.625147 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.625104    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerName="revoke-keys"
Jul 17 18:09:56.625147 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.625116    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerName="revoke-keys"
Jul 17 18:09:56.632585 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.631012    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"
Jul 17 18:09:56.642403 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.642358    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"]
Jul 17 18:09:56.725173 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.725131    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp\" (UID: \"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"
Jul 17 18:09:56.826432 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.826393    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp\" (UID: \"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"
Jul 17 18:09:56.827098 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.827075    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp\" (UID: \"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"
Jul 17 18:09:56.944819 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:56.944781    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"
Jul 17 18:09:57.086427 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:57.086383    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"]
Jul 17 18:09:57.087915 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:09:57.087887    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd07e02_8b8b_4bcf_9aed_ded3e8c30c47.slice/crio-e9cd3b1a97208ccf8cf881eb131279000dbe02eec096cb46f91d723458932908 WatchSource:0}: Error finding container e9cd3b1a97208ccf8cf881eb131279000dbe02eec096cb46f91d723458932908: Status 404 returned error can't find the container with id e9cd3b1a97208ccf8cf881eb131279000dbe02eec096cb46f91d723458932908
Jul 17 18:09:57.613651 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:57.613545    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" event={"ID":"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47","Type":"ContainerStarted","Data":"360c1c7f141fdaa49f8eacbb9ef7223955136821e2c0816014d1b479ce8544d8"}
Jul 17 18:09:57.613651 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:57.613603    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" event={"ID":"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47","Type":"ContainerStarted","Data":"e9cd3b1a97208ccf8cf881eb131279000dbe02eec096cb46f91d723458932908"}
Jul 17 18:09:57.632472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:57.632410    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" podStartSLOduration=1.632393054 podStartE2EDuration="1.632393054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:09:57.629135008 +0000 UTC m=+2769.634490863" watchObservedRunningTime="2026-07-17 18:09:57.632393054 +0000 UTC m=+2769.637748914"
Jul 17 18:09:58.471314 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.471273    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"]
Jul 17 18:09:58.471679 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.471665    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerName="revoke-keys"
Jul 17 18:09:58.471724 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.471680    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerName="revoke-keys"
Jul 17 18:09:58.471757 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.471752    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ef08ee4-98b8-4428-a3da-2a1bef563a09" containerName="revoke-keys"
Jul 17 18:09:58.474761 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.474741    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"
Jul 17 18:09:58.482648 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.482514    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"]
Jul 17 18:09:58.512652 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.512624    2572 scope.go:117] "RemoveContainer" containerID="017b6147317ec6091dc91aaeb605f5067bc55533d664e1adfde429ae6e368a03"
Jul 17 18:09:58.621067 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.621016    2572 generic.go:356] "Generic (PLEG): container finished" podID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerID="360c1c7f141fdaa49f8eacbb9ef7223955136821e2c0816014d1b479ce8544d8" exitCode=6
Jul 17 18:09:58.621561 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.621107    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" event={"ID":"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47","Type":"ContainerDied","Data":"360c1c7f141fdaa49f8eacbb9ef7223955136821e2c0816014d1b479ce8544d8"}
Jul 17 18:09:58.621561 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.621306    2572 scope.go:117] "RemoveContainer" containerID="360c1c7f141fdaa49f8eacbb9ef7223955136821e2c0816014d1b479ce8544d8"
Jul 17 18:09:58.643474 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.643443    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82efeca4-39aa-4b8f-ae21-217e5a2ef18f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj\" (UID: \"82efeca4-39aa-4b8f-ae21-217e5a2ef18f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"
Jul 17 18:09:58.744217 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.744196    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82efeca4-39aa-4b8f-ae21-217e5a2ef18f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj\" (UID: \"82efeca4-39aa-4b8f-ae21-217e5a2ef18f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"
Jul 17 18:09:58.744875 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.744856    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82efeca4-39aa-4b8f-ae21-217e5a2ef18f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj\" (UID: \"82efeca4-39aa-4b8f-ae21-217e5a2ef18f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"
Jul 17 18:09:58.787479 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.787438    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"
Jul 17 18:09:58.934605 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:58.934392    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"]
Jul 17 18:09:58.936762 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:09:58.936730    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82efeca4_39aa_4b8f_ae21_217e5a2ef18f.slice/crio-75a8d755df5b3a5a3d1f93e781b5cfa682182c8b2147ad781a4171a8072a9843 WatchSource:0}: Error finding container 75a8d755df5b3a5a3d1f93e781b5cfa682182c8b2147ad781a4171a8072a9843: Status 404 returned error can't find the container with id 75a8d755df5b3a5a3d1f93e781b5cfa682182c8b2147ad781a4171a8072a9843
Jul 17 18:09:59.629338 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.629292    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" event={"ID":"82efeca4-39aa-4b8f-ae21-217e5a2ef18f","Type":"ContainerStarted","Data":"fc3a0b4253bb5cf2705b081ab70ffdbe3d4afd000ba9ac7052ab8a42e9e6320e"}
Jul 17 18:09:59.629856 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.629370    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" event={"ID":"82efeca4-39aa-4b8f-ae21-217e5a2ef18f","Type":"ContainerStarted","Data":"75a8d755df5b3a5a3d1f93e781b5cfa682182c8b2147ad781a4171a8072a9843"}
Jul 17 18:09:59.631399 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.631369    2572 generic.go:356] "Generic (PLEG): container finished" podID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerID="014d7687eb6fbdf30bfa81c831e7dff898ad7f4aa31038e9eecc95b6343d65f8" exitCode=6
Jul 17 18:09:59.631561 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.631456    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" event={"ID":"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47","Type":"ContainerDied","Data":"014d7687eb6fbdf30bfa81c831e7dff898ad7f4aa31038e9eecc95b6343d65f8"}
Jul 17 18:09:59.631561 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.631539    2572 scope.go:117] "RemoveContainer" containerID="360c1c7f141fdaa49f8eacbb9ef7223955136821e2c0816014d1b479ce8544d8"
Jul 17 18:09:59.631713 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.631697    2572 scope.go:117] "RemoveContainer" containerID="014d7687eb6fbdf30bfa81c831e7dff898ad7f4aa31038e9eecc95b6343d65f8"
Jul 17 18:09:59.631966 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:59.631942    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp_odh-ai-gateway-infra(ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47"
Jul 17 18:09:59.633685 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.633662    2572 generic.go:356] "Generic (PLEG): container finished" podID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerID="3c9362db44da0f3997c86a22d74ee43f2a811c17e50e6418c1e82488bc034bf4" exitCode=6
Jul 17 18:09:59.633772 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.633695    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb" event={"ID":"c56a6a7e-2f5c-410a-8b76-1ab361c0af05","Type":"ContainerDied","Data":"3c9362db44da0f3997c86a22d74ee43f2a811c17e50e6418c1e82488bc034bf4"}
Jul 17 18:09:59.634002 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.633989    2572 scope.go:117] "RemoveContainer" containerID="3c9362db44da0f3997c86a22d74ee43f2a811c17e50e6418c1e82488bc034bf4"
Jul 17 18:09:59.634172 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:09:59.634157    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb_odh-ai-gateway-infra(c56a6a7e-2f5c-410a-8b76-1ab361c0af05)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05"
Jul 17 18:09:59.643378 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.643355    2572 scope.go:117] "RemoveContainer" containerID="017b6147317ec6091dc91aaeb605f5067bc55533d664e1adfde429ae6e368a03"
Jul 17 18:09:59.649268 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:09:59.649219    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" podStartSLOduration=1.64920291 podStartE2EDuration="1.64920291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:09:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:09:59.648286157 +0000 UTC m=+2771.653642012" watchObservedRunningTime="2026-07-17 18:09:59.64920291 +0000 UTC m=+2771.654558845"
Jul 17 18:10:00.641949 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:00.641912    2572 generic.go:356] "Generic (PLEG): container finished" podID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerID="fc3a0b4253bb5cf2705b081ab70ffdbe3d4afd000ba9ac7052ab8a42e9e6320e" exitCode=6
Jul 17 18:10:00.642432 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:00.641999    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" event={"ID":"82efeca4-39aa-4b8f-ae21-217e5a2ef18f","Type":"ContainerDied","Data":"fc3a0b4253bb5cf2705b081ab70ffdbe3d4afd000ba9ac7052ab8a42e9e6320e"}
Jul 17 18:10:00.642432 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:00.642205    2572 scope.go:117] "RemoveContainer" containerID="fc3a0b4253bb5cf2705b081ab70ffdbe3d4afd000ba9ac7052ab8a42e9e6320e"
Jul 17 18:10:00.643949 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:00.643931    2572 scope.go:117] "RemoveContainer" containerID="014d7687eb6fbdf30bfa81c831e7dff898ad7f4aa31038e9eecc95b6343d65f8"
Jul 17 18:10:00.644097 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:10:00.644078    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp_odh-ai-gateway-infra(ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47"
Jul 17 18:10:00.677425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:00.677392    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"]
Jul 17 18:10:00.821468 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:00.821435    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"
Jul 17 18:10:00.965695 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:00.965584    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c56a6a7e-2f5c-410a-8b76-1ab361c0af05-maas-api-service-ca\") pod \"c56a6a7e-2f5c-410a-8b76-1ab361c0af05\" (UID: \"c56a6a7e-2f5c-410a-8b76-1ab361c0af05\") "
Jul 17 18:10:00.965994 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:00.965968    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c56a6a7e-2f5c-410a-8b76-1ab361c0af05-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c56a6a7e-2f5c-410a-8b76-1ab361c0af05" (UID: "c56a6a7e-2f5c-410a-8b76-1ab361c0af05"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:10:01.067117 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.067079    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c56a6a7e-2f5c-410a-8b76-1ab361c0af05-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:10:01.651635 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.651536    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"
Jul 17 18:10:01.651635 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.651544    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb" event={"ID":"c56a6a7e-2f5c-410a-8b76-1ab361c0af05","Type":"ContainerDied","Data":"2372ad92c0651d94503aa4c6dfee93756866883e0728a7333a2faabe5379e594"}
Jul 17 18:10:01.651635 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.651618    2572 scope.go:117] "RemoveContainer" containerID="3c9362db44da0f3997c86a22d74ee43f2a811c17e50e6418c1e82488bc034bf4"
Jul 17 18:10:01.653720 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.653690    2572 generic.go:356] "Generic (PLEG): container finished" podID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerID="71613da560bbe21f059c042249697a704dc8497003e23a82f13d59557fa65b05" exitCode=6
Jul 17 18:10:01.653840 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.653813    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" event={"ID":"82efeca4-39aa-4b8f-ae21-217e5a2ef18f","Type":"ContainerDied","Data":"71613da560bbe21f059c042249697a704dc8497003e23a82f13d59557fa65b05"}
Jul 17 18:10:01.654007 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.653989    2572 scope.go:117] "RemoveContainer" containerID="71613da560bbe21f059c042249697a704dc8497003e23a82f13d59557fa65b05"
Jul 17 18:10:01.654243 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:10:01.654222    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj_odh-ai-gateway-infra(82efeca4-39aa-4b8f-ae21-217e5a2ef18f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f"
Jul 17 18:10:01.661890 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.661867    2572 scope.go:117] "RemoveContainer" containerID="fc3a0b4253bb5cf2705b081ab70ffdbe3d4afd000ba9ac7052ab8a42e9e6320e"
Jul 17 18:10:01.687752 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.687708    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"]
Jul 17 18:10:01.693565 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:01.693528    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-2dgzb"]
Jul 17 18:10:02.516280 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:02.516249    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" path="/var/lib/kubelet/pods/c56a6a7e-2f5c-410a-8b76-1ab361c0af05/volumes"
Jul 17 18:10:02.662039 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:02.662010    2572 scope.go:117] "RemoveContainer" containerID="71613da560bbe21f059c042249697a704dc8497003e23a82f13d59557fa65b05"
Jul 17 18:10:02.662467 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:10:02.662226    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj_odh-ai-gateway-infra(82efeca4-39aa-4b8f-ae21-217e5a2ef18f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f"
Jul 17 18:10:15.509814 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:15.509725    2572 scope.go:117] "RemoveContainer" containerID="014d7687eb6fbdf30bfa81c831e7dff898ad7f4aa31038e9eecc95b6343d65f8"
Jul 17 18:10:16.510631 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:16.510540    2572 scope.go:117] "RemoveContainer" containerID="71613da560bbe21f059c042249697a704dc8497003e23a82f13d59557fa65b05"
Jul 17 18:10:16.740479 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:16.740449    2572 generic.go:356] "Generic (PLEG): container finished" podID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerID="52b09313fdfb60dd23441d77a4d0c96e548776544f45f07cc151a9d022449edf" exitCode=6
Jul 17 18:10:16.740645 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:16.740522    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" event={"ID":"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47","Type":"ContainerDied","Data":"52b09313fdfb60dd23441d77a4d0c96e548776544f45f07cc151a9d022449edf"}
Jul 17 18:10:16.740645 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:16.740591    2572 scope.go:117] "RemoveContainer" containerID="014d7687eb6fbdf30bfa81c831e7dff898ad7f4aa31038e9eecc95b6343d65f8"
Jul 17 18:10:16.740917 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:16.740899    2572 scope.go:117] "RemoveContainer" containerID="52b09313fdfb60dd23441d77a4d0c96e548776544f45f07cc151a9d022449edf"
Jul 17 18:10:16.741152 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:10:16.741132    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp_odh-ai-gateway-infra(ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47"
Jul 17 18:10:17.751848 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:17.751815    2572 generic.go:356] "Generic (PLEG): container finished" podID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerID="f1acb005244a386209ff97469ecae84f34fc51019f15d871a826895aa5febf5e" exitCode=6
Jul 17 18:10:17.752340 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:17.751890    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" event={"ID":"82efeca4-39aa-4b8f-ae21-217e5a2ef18f","Type":"ContainerDied","Data":"f1acb005244a386209ff97469ecae84f34fc51019f15d871a826895aa5febf5e"}
Jul 17 18:10:17.752340 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:17.751958    2572 scope.go:117] "RemoveContainer" containerID="71613da560bbe21f059c042249697a704dc8497003e23a82f13d59557fa65b05"
Jul 17 18:10:17.752340 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:17.752290    2572 scope.go:117] "RemoveContainer" containerID="f1acb005244a386209ff97469ecae84f34fc51019f15d871a826895aa5febf5e"
Jul 17 18:10:17.752580 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:10:17.752558    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj_odh-ai-gateway-infra(82efeca4-39aa-4b8f-ae21-217e5a2ef18f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f"
Jul 17 18:10:17.769014 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:17.768983    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"]
Jul 17 18:10:17.908608 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:17.908575    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"
Jul 17 18:10:18.018804 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.018706    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47-maas-api-service-ca\") pod \"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47\" (UID: \"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47\") "
Jul 17 18:10:18.019085 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.019059    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" (UID: "ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:10:18.119976 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.119938    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:10:18.761091 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.761054    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp" event={"ID":"ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47","Type":"ContainerDied","Data":"e9cd3b1a97208ccf8cf881eb131279000dbe02eec096cb46f91d723458932908"}
Jul 17 18:10:18.761091 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.761086    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"
Jul 17 18:10:18.761672 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.761126    2572 scope.go:117] "RemoveContainer" containerID="52b09313fdfb60dd23441d77a4d0c96e548776544f45f07cc151a9d022449edf"
Jul 17 18:10:18.780885 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.780798    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"]
Jul 17 18:10:18.784536 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.784501    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"]
Jul 17 18:10:18.787306 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.787280    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-nlpnp"]
Jul 17 18:10:18.929992 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:18.929963    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"
Jul 17 18:10:19.026905 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:19.026867    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82efeca4-39aa-4b8f-ae21-217e5a2ef18f-maas-api-service-ca\") pod \"82efeca4-39aa-4b8f-ae21-217e5a2ef18f\" (UID: \"82efeca4-39aa-4b8f-ae21-217e5a2ef18f\") "
Jul 17 18:10:19.027231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:19.027209    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82efeca4-39aa-4b8f-ae21-217e5a2ef18f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82efeca4-39aa-4b8f-ae21-217e5a2ef18f" (UID: "82efeca4-39aa-4b8f-ae21-217e5a2ef18f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:10:19.128195 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:19.128150    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82efeca4-39aa-4b8f-ae21-217e5a2ef18f-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:10:19.767845 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:19.767815    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"
Jul 17 18:10:19.768333 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:19.767856    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj" event={"ID":"82efeca4-39aa-4b8f-ae21-217e5a2ef18f","Type":"ContainerDied","Data":"75a8d755df5b3a5a3d1f93e781b5cfa682182c8b2147ad781a4171a8072a9843"}
Jul 17 18:10:19.768333 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:19.767925    2572 scope.go:117] "RemoveContainer" containerID="f1acb005244a386209ff97469ecae84f34fc51019f15d871a826895aa5febf5e"
Jul 17 18:10:19.806786 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:19.806749    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"]
Jul 17 18:10:19.822635 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:19.822598    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nsrsj"]
Jul 17 18:10:20.521260 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:20.516159    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" path="/var/lib/kubelet/pods/82efeca4-39aa-4b8f-ae21-217e5a2ef18f/volumes"
Jul 17 18:10:20.521260 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:20.516792    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" path="/var/lib/kubelet/pods/ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47/volumes"
Jul 17 18:10:56.671563 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671524    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"]
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671863    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671873    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671882    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671888    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671898    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671903    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671913    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671918    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671929    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671934    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671940    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671945    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671950    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671955    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671964    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerName="revoke-keys"
Jul 17 18:10:56.672003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.671969    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerName="revoke-keys"
Jul 17 18:10:56.672472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.672021    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerName="revoke-keys"
Jul 17 18:10:56.672472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.672028    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerName="revoke-keys"
Jul 17 18:10:56.672472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.672033    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerName="revoke-keys"
Jul 17 18:10:56.672472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.672039    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="82efeca4-39aa-4b8f-ae21-217e5a2ef18f" containerName="revoke-keys"
Jul 17 18:10:56.672472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.672044    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerName="revoke-keys"
Jul 17 18:10:56.672472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.672049    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerName="revoke-keys"
Jul 17 18:10:56.672472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.672056    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerName="revoke-keys"
Jul 17 18:10:56.672472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.672062    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerName="revoke-keys"
Jul 17 18:10:56.676481 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.676464    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"
Jul 17 18:10:56.679535 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.679511    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:10:56.679733 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.679555    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:10:56.684699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.684673    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"]
Jul 17 18:10:56.763517 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.763448    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b9713b-b12e-48b8-95c8-4d1eb796ad50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-9p664\" (UID: \"11b9713b-b12e-48b8-95c8-4d1eb796ad50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"
Jul 17 18:10:56.864411 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.864370    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b9713b-b12e-48b8-95c8-4d1eb796ad50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-9p664\" (UID: \"11b9713b-b12e-48b8-95c8-4d1eb796ad50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"
Jul 17 18:10:56.864994 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.864975    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b9713b-b12e-48b8-95c8-4d1eb796ad50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-9p664\" (UID: \"11b9713b-b12e-48b8-95c8-4d1eb796ad50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"
Jul 17 18:10:56.988340 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:56.988239    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"
Jul 17 18:10:57.152280 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:57.152249    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"]
Jul 17 18:10:57.153892 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:10:57.153867    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b9713b_b12e_48b8_95c8_4d1eb796ad50.slice/crio-fdd2ded48a30051eb0db833219661c9bba94a21ec0fa2be30ee5c671e660f6f3 WatchSource:0}: Error finding container fdd2ded48a30051eb0db833219661c9bba94a21ec0fa2be30ee5c671e660f6f3: Status 404 returned error can't find the container with id fdd2ded48a30051eb0db833219661c9bba94a21ec0fa2be30ee5c671e660f6f3
Jul 17 18:10:57.976006 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:57.975968    2572 generic.go:356] "Generic (PLEG): container finished" podID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerID="3be1c163b71fdcb4996cbed95746970858a815187e9b95732355f239ff3c4694" exitCode=6
Jul 17 18:10:57.976453 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:57.976056    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" event={"ID":"11b9713b-b12e-48b8-95c8-4d1eb796ad50","Type":"ContainerDied","Data":"3be1c163b71fdcb4996cbed95746970858a815187e9b95732355f239ff3c4694"}
Jul 17 18:10:57.976453 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:57.976118    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" event={"ID":"11b9713b-b12e-48b8-95c8-4d1eb796ad50","Type":"ContainerStarted","Data":"fdd2ded48a30051eb0db833219661c9bba94a21ec0fa2be30ee5c671e660f6f3"}
Jul 17 18:10:57.976453 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:57.976266    2572 scope.go:117] "RemoveContainer" containerID="3be1c163b71fdcb4996cbed95746970858a815187e9b95732355f239ff3c4694"
Jul 17 18:10:58.526508 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.526435    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"]
Jul 17 18:10:58.526997 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.526975    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerName="revoke-keys"
Jul 17 18:10:58.527107 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.527001    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd07e02-8b8b-4bcf-9aed-ded3e8c30c47" containerName="revoke-keys"
Jul 17 18:10:58.527161 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.527114    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c56a6a7e-2f5c-410a-8b76-1ab361c0af05" containerName="revoke-keys"
Jul 17 18:10:58.530174 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.530151    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"
Jul 17 18:10:58.538484 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.538431    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"]
Jul 17 18:10:58.582708 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.582664    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f96720e-cdc2-4471-804b-0cc81c0637b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6\" (UID: \"1f96720e-cdc2-4471-804b-0cc81c0637b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"
Jul 17 18:10:58.683923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.683877    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f96720e-cdc2-4471-804b-0cc81c0637b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6\" (UID: \"1f96720e-cdc2-4471-804b-0cc81c0637b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"
Jul 17 18:10:58.684569 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.684549    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f96720e-cdc2-4471-804b-0cc81c0637b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6\" (UID: \"1f96720e-cdc2-4471-804b-0cc81c0637b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"
Jul 17 18:10:58.843864 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.843809    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"
Jul 17 18:10:58.985085 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.985040    2572 generic.go:356] "Generic (PLEG): container finished" podID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerID="b12d3ca0c43e8baecbc5f46949f72dd13ab8a2c797b2c94fd9a3866dc39ac9fc" exitCode=6
Jul 17 18:10:58.985589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.985126    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" event={"ID":"11b9713b-b12e-48b8-95c8-4d1eb796ad50","Type":"ContainerDied","Data":"b12d3ca0c43e8baecbc5f46949f72dd13ab8a2c797b2c94fd9a3866dc39ac9fc"}
Jul 17 18:10:58.985589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.985195    2572 scope.go:117] "RemoveContainer" containerID="3be1c163b71fdcb4996cbed95746970858a815187e9b95732355f239ff3c4694"
Jul 17 18:10:58.985589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.985405    2572 scope.go:117] "RemoveContainer" containerID="b12d3ca0c43e8baecbc5f46949f72dd13ab8a2c797b2c94fd9a3866dc39ac9fc"
Jul 17 18:10:58.985790 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:10:58.985702    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-9p664_odh-ai-gateway-infra(11b9713b-b12e-48b8-95c8-4d1eb796ad50)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50"
Jul 17 18:10:58.987148 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:58.987127    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"]
Jul 17 18:10:58.989559 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:10:58.989528    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f96720e_cdc2_4471_804b_0cc81c0637b0.slice/crio-869bce96f52b5e0bf552b4ba6f3a03250b2784586e13769d86b4f5280183223d WatchSource:0}: Error finding container 869bce96f52b5e0bf552b4ba6f3a03250b2784586e13769d86b4f5280183223d: Status 404 returned error can't find the container with id 869bce96f52b5e0bf552b4ba6f3a03250b2784586e13769d86b4f5280183223d
Jul 17 18:10:59.992692 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:59.992588    2572 generic.go:356] "Generic (PLEG): container finished" podID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerID="fbc0cfbdd0d389d4615dea120c7d713eabc2376fe4f03f6c18c946bf3ecbd309" exitCode=6
Jul 17 18:10:59.993167 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:59.992678    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" event={"ID":"1f96720e-cdc2-4471-804b-0cc81c0637b0","Type":"ContainerDied","Data":"fbc0cfbdd0d389d4615dea120c7d713eabc2376fe4f03f6c18c946bf3ecbd309"}
Jul 17 18:10:59.993167 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:59.992751    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" event={"ID":"1f96720e-cdc2-4471-804b-0cc81c0637b0","Type":"ContainerStarted","Data":"869bce96f52b5e0bf552b4ba6f3a03250b2784586e13769d86b4f5280183223d"}
Jul 17 18:10:59.993167 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:59.992901    2572 scope.go:117] "RemoveContainer" containerID="fbc0cfbdd0d389d4615dea120c7d713eabc2376fe4f03f6c18c946bf3ecbd309"
Jul 17 18:10:59.994651 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:10:59.994632    2572 scope.go:117] "RemoveContainer" containerID="b12d3ca0c43e8baecbc5f46949f72dd13ab8a2c797b2c94fd9a3866dc39ac9fc"
Jul 17 18:10:59.994798 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:10:59.994780    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-9p664_odh-ai-gateway-infra(11b9713b-b12e-48b8-95c8-4d1eb796ad50)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50"
Jul 17 18:11:01.002156 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:01.002058    2572 generic.go:356] "Generic (PLEG): container finished" podID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerID="0567fd76e8b130e291f0bcf777c0ac00bb3e457e9861e2df404d52e62804e3dd" exitCode=6
Jul 17 18:11:01.002645 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:01.002147    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" event={"ID":"1f96720e-cdc2-4471-804b-0cc81c0637b0","Type":"ContainerDied","Data":"0567fd76e8b130e291f0bcf777c0ac00bb3e457e9861e2df404d52e62804e3dd"}
Jul 17 18:11:01.002645 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:01.002221    2572 scope.go:117] "RemoveContainer" containerID="fbc0cfbdd0d389d4615dea120c7d713eabc2376fe4f03f6c18c946bf3ecbd309"
Jul 17 18:11:01.002645 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:01.002468    2572 scope.go:117] "RemoveContainer" containerID="0567fd76e8b130e291f0bcf777c0ac00bb3e457e9861e2df404d52e62804e3dd"
Jul 17 18:11:01.002771 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:01.002727    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6_odh-ai-gateway-infra(1f96720e-cdc2-4471-804b-0cc81c0637b0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0"
Jul 17 18:11:02.009608 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:02.009578    2572 scope.go:117] "RemoveContainer" containerID="0567fd76e8b130e291f0bcf777c0ac00bb3e457e9861e2df404d52e62804e3dd"
Jul 17 18:11:02.010032 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:02.009809    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6_odh-ai-gateway-infra(1f96720e-cdc2-4471-804b-0cc81c0637b0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0"
Jul 17 18:11:14.515485 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:14.515452    2572 scope.go:117] "RemoveContainer" containerID="0567fd76e8b130e291f0bcf777c0ac00bb3e457e9861e2df404d52e62804e3dd"
Jul 17 18:11:14.515947 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:14.515669    2572 scope.go:117] "RemoveContainer" containerID="b12d3ca0c43e8baecbc5f46949f72dd13ab8a2c797b2c94fd9a3866dc39ac9fc"
Jul 17 18:11:15.087108 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:15.087008    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" event={"ID":"1f96720e-cdc2-4471-804b-0cc81c0637b0","Type":"ContainerStarted","Data":"c11f23b9b7711486286619ba06981d4285c6d0a31beaf74b76ee653cac9822ec"}
Jul 17 18:11:15.088744 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:15.088716    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" event={"ID":"11b9713b-b12e-48b8-95c8-4d1eb796ad50","Type":"ContainerStarted","Data":"c8838c73b75ff5a2cc4cec74e9025a12ec88991ed883730769b0ee95380eaf42"}
Jul 17 18:11:15.123345 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:15.123278    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" podStartSLOduration=17.123256106 podStartE2EDuration="17.123256106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:10:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:11:15.121143012 +0000 UTC m=+2847.126498869" watchObservedRunningTime="2026-07-17 18:11:15.123256106 +0000 UTC m=+2847.128611964"
Jul 17 18:11:15.148744 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:15.148693    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" podStartSLOduration=19.148674965 podStartE2EDuration="19.148674965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:11:15.147878137 +0000 UTC m=+2847.153233991" watchObservedRunningTime="2026-07-17 18:11:15.148674965 +0000 UTC m=+2847.154030798"
Jul 17 18:11:16.096851 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.096814    2572 generic.go:356] "Generic (PLEG): container finished" podID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerID="c11f23b9b7711486286619ba06981d4285c6d0a31beaf74b76ee653cac9822ec" exitCode=6
Jul 17 18:11:16.097363 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.096905    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" event={"ID":"1f96720e-cdc2-4471-804b-0cc81c0637b0","Type":"ContainerDied","Data":"c11f23b9b7711486286619ba06981d4285c6d0a31beaf74b76ee653cac9822ec"}
Jul 17 18:11:16.097363 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.096967    2572 scope.go:117] "RemoveContainer" containerID="0567fd76e8b130e291f0bcf777c0ac00bb3e457e9861e2df404d52e62804e3dd"
Jul 17 18:11:16.097363 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.097303    2572 scope.go:117] "RemoveContainer" containerID="c11f23b9b7711486286619ba06981d4285c6d0a31beaf74b76ee653cac9822ec"
Jul 17 18:11:16.097609 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:16.097584    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6_odh-ai-gateway-infra(1f96720e-cdc2-4471-804b-0cc81c0637b0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0"
Jul 17 18:11:16.099095 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.099068    2572 generic.go:356] "Generic (PLEG): container finished" podID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerID="c8838c73b75ff5a2cc4cec74e9025a12ec88991ed883730769b0ee95380eaf42" exitCode=6
Jul 17 18:11:16.099209 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.099145    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" event={"ID":"11b9713b-b12e-48b8-95c8-4d1eb796ad50","Type":"ContainerDied","Data":"c8838c73b75ff5a2cc4cec74e9025a12ec88991ed883730769b0ee95380eaf42"}
Jul 17 18:11:16.099443 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.099429    2572 scope.go:117] "RemoveContainer" containerID="c8838c73b75ff5a2cc4cec74e9025a12ec88991ed883730769b0ee95380eaf42"
Jul 17 18:11:16.099683 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:16.099662    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-9p664_odh-ai-gateway-infra(11b9713b-b12e-48b8-95c8-4d1eb796ad50)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50"
Jul 17 18:11:16.109163 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.109145    2572 scope.go:117] "RemoveContainer" containerID="b12d3ca0c43e8baecbc5f46949f72dd13ab8a2c797b2c94fd9a3866dc39ac9fc"
Jul 17 18:11:16.150754 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.150706    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"]
Jul 17 18:11:16.159001 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:16.158968    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"]
Jul 17 18:11:17.254092 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:17.254062    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"
Jul 17 18:11:17.269594 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:17.269568    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"
Jul 17 18:11:17.350006 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:17.349971    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f96720e-cdc2-4471-804b-0cc81c0637b0-maas-api-service-ca\") pod \"1f96720e-cdc2-4471-804b-0cc81c0637b0\" (UID: \"1f96720e-cdc2-4471-804b-0cc81c0637b0\") "
Jul 17 18:11:17.350213 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:17.350048    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b9713b-b12e-48b8-95c8-4d1eb796ad50-maas-api-service-ca\") pod \"11b9713b-b12e-48b8-95c8-4d1eb796ad50\" (UID: \"11b9713b-b12e-48b8-95c8-4d1eb796ad50\") "
Jul 17 18:11:17.350400 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:17.350379    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11b9713b-b12e-48b8-95c8-4d1eb796ad50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11b9713b-b12e-48b8-95c8-4d1eb796ad50" (UID: "11b9713b-b12e-48b8-95c8-4d1eb796ad50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:11:17.350467 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:17.350381    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f96720e-cdc2-4471-804b-0cc81c0637b0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f96720e-cdc2-4471-804b-0cc81c0637b0" (UID: "1f96720e-cdc2-4471-804b-0cc81c0637b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:11:17.451552 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:17.451414    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b9713b-b12e-48b8-95c8-4d1eb796ad50-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:11:17.451552 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:17.451452    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f96720e-cdc2-4471-804b-0cc81c0637b0-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:11:18.116431 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.116399    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"
Jul 17 18:11:18.116431 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.116420    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664" event={"ID":"11b9713b-b12e-48b8-95c8-4d1eb796ad50","Type":"ContainerDied","Data":"fdd2ded48a30051eb0db833219661c9bba94a21ec0fa2be30ee5c671e660f6f3"}
Jul 17 18:11:18.116737 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.116507    2572 scope.go:117] "RemoveContainer" containerID="c8838c73b75ff5a2cc4cec74e9025a12ec88991ed883730769b0ee95380eaf42"
Jul 17 18:11:18.118085 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.118061    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6" event={"ID":"1f96720e-cdc2-4471-804b-0cc81c0637b0","Type":"ContainerDied","Data":"869bce96f52b5e0bf552b4ba6f3a03250b2784586e13769d86b4f5280183223d"}
Jul 17 18:11:18.118209 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.118095    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"
Jul 17 18:11:18.133077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.133055    2572 scope.go:117] "RemoveContainer" containerID="c11f23b9b7711486286619ba06981d4285c6d0a31beaf74b76ee653cac9822ec"
Jul 17 18:11:18.148777 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.148745    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"]
Jul 17 18:11:18.152623 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.152597    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-9p664"]
Jul 17 18:11:18.163561 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.163531    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"]
Jul 17 18:11:18.167538 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.167510    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-ct8j6"]
Jul 17 18:11:18.516140 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.516054    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" path="/var/lib/kubelet/pods/11b9713b-b12e-48b8-95c8-4d1eb796ad50/volumes"
Jul 17 18:11:18.516523 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:18.516394    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" path="/var/lib/kubelet/pods/1f96720e-cdc2-4471-804b-0cc81c0637b0/volumes"
Jul 17 18:11:29.994063 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994026    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"]
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994333    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994343    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994355    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994360    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994376    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994383    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994394    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994412    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994480    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994506    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994513    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerName="revoke-keys"
Jul 17 18:11:29.994602 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.994522    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerName="revoke-keys"
Jul 17 18:11:29.997541 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:29.997525    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"
Jul 17 18:11:30.000400 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:30.000374    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:11:30.000545 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:30.000430    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:11:30.006707 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:30.006666    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"]
Jul 17 18:11:30.163262 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:30.163218    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14b97b0-dec3-48d9-b61d-18236b385bdc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-l9d98\" (UID: \"b14b97b0-dec3-48d9-b61d-18236b385bdc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"
Jul 17 18:11:30.264732 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:30.264631    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14b97b0-dec3-48d9-b61d-18236b385bdc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-l9d98\" (UID: \"b14b97b0-dec3-48d9-b61d-18236b385bdc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"
Jul 17 18:11:30.265289 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:30.265268    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14b97b0-dec3-48d9-b61d-18236b385bdc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-l9d98\" (UID: \"b14b97b0-dec3-48d9-b61d-18236b385bdc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"
Jul 17 18:11:30.308678 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:30.308640    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"
Jul 17 18:11:30.459352 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:30.459315    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"]
Jul 17 18:11:30.461164 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:11:30.461136    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14b97b0_dec3_48d9_b61d_18236b385bdc.slice/crio-fe72e72f09d4bbe1b8ac23a39463dd6c9f2a4bc170a7676ec7a3b8f26abd6634 WatchSource:0}: Error finding container fe72e72f09d4bbe1b8ac23a39463dd6c9f2a4bc170a7676ec7a3b8f26abd6634: Status 404 returned error can't find the container with id fe72e72f09d4bbe1b8ac23a39463dd6c9f2a4bc170a7676ec7a3b8f26abd6634
Jul 17 18:11:31.196930 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:31.196893    2572 generic.go:356] "Generic (PLEG): container finished" podID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerID="988630bfa7fa37558f8fc84eedb0993421ff571e001504e1fcfa1f90d99fc374" exitCode=6
Jul 17 18:11:31.197345 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:31.196981    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98" event={"ID":"b14b97b0-dec3-48d9-b61d-18236b385bdc","Type":"ContainerDied","Data":"988630bfa7fa37558f8fc84eedb0993421ff571e001504e1fcfa1f90d99fc374"}
Jul 17 18:11:31.197345 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:31.197038    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98" event={"ID":"b14b97b0-dec3-48d9-b61d-18236b385bdc","Type":"ContainerStarted","Data":"fe72e72f09d4bbe1b8ac23a39463dd6c9f2a4bc170a7676ec7a3b8f26abd6634"}
Jul 17 18:11:31.197345 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:31.197175    2572 scope.go:117] "RemoveContainer" containerID="988630bfa7fa37558f8fc84eedb0993421ff571e001504e1fcfa1f90d99fc374"
Jul 17 18:11:32.204363 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:32.204270    2572 generic.go:356] "Generic (PLEG): container finished" podID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerID="fe65a97170bcd0d79dd712289d8e53749d49554334c014debdbb6e27c289cd58" exitCode=6
Jul 17 18:11:32.204781 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:32.204353    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98" event={"ID":"b14b97b0-dec3-48d9-b61d-18236b385bdc","Type":"ContainerDied","Data":"fe65a97170bcd0d79dd712289d8e53749d49554334c014debdbb6e27c289cd58"}
Jul 17 18:11:32.204781 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:32.204455    2572 scope.go:117] "RemoveContainer" containerID="988630bfa7fa37558f8fc84eedb0993421ff571e001504e1fcfa1f90d99fc374"
Jul 17 18:11:32.204781 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:32.204654    2572 scope.go:117] "RemoveContainer" containerID="fe65a97170bcd0d79dd712289d8e53749d49554334c014debdbb6e27c289cd58"
Jul 17 18:11:32.204917 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:32.204878    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-l9d98_odh-ai-gateway-infra(b14b97b0-dec3-48d9-b61d-18236b385bdc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc"
Jul 17 18:11:33.211467 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:33.211440    2572 scope.go:117] "RemoveContainer" containerID="fe65a97170bcd0d79dd712289d8e53749d49554334c014debdbb6e27c289cd58"
Jul 17 18:11:33.213875 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:33.211734    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-l9d98_odh-ai-gateway-infra(b14b97b0-dec3-48d9-b61d-18236b385bdc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc"
Jul 17 18:11:34.451780 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.451740    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"]
Jul 17 18:11:34.452179 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.452086    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerName="revoke-keys"
Jul 17 18:11:34.452179 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.452098    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerName="revoke-keys"
Jul 17 18:11:34.452179 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.452110    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerName="revoke-keys"
Jul 17 18:11:34.452179 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.452118    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerName="revoke-keys"
Jul 17 18:11:34.452179 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.452176    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b9713b-b12e-48b8-95c8-4d1eb796ad50" containerName="revoke-keys"
Jul 17 18:11:34.452342 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.452185    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f96720e-cdc2-4471-804b-0cc81c0637b0" containerName="revoke-keys"
Jul 17 18:11:34.455153 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.455136    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"
Jul 17 18:11:34.473810 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.473771    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"]
Jul 17 18:11:34.601904 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.601858    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636f82e1-6c30-4536-af38-bd936909448d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt\" (UID: \"636f82e1-6c30-4536-af38-bd936909448d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"
Jul 17 18:11:34.703340 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.703235    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636f82e1-6c30-4536-af38-bd936909448d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt\" (UID: \"636f82e1-6c30-4536-af38-bd936909448d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"
Jul 17 18:11:34.703930 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.703910    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636f82e1-6c30-4536-af38-bd936909448d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt\" (UID: \"636f82e1-6c30-4536-af38-bd936909448d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"
Jul 17 18:11:34.765128 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.765096    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"
Jul 17 18:11:34.909070 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:34.909036    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"]
Jul 17 18:11:34.910441 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:11:34.910385    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636f82e1_6c30_4536_af38_bd936909448d.slice/crio-8ceca2cc3b5c7c29a2ec382af163e9a69513392b28af38f226ed41b41c253099 WatchSource:0}: Error finding container 8ceca2cc3b5c7c29a2ec382af163e9a69513392b28af38f226ed41b41c253099: Status 404 returned error can't find the container with id 8ceca2cc3b5c7c29a2ec382af163e9a69513392b28af38f226ed41b41c253099
Jul 17 18:11:35.224438 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:35.224402    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt" event={"ID":"636f82e1-6c30-4536-af38-bd936909448d","Type":"ContainerStarted","Data":"8ceca2cc3b5c7c29a2ec382af163e9a69513392b28af38f226ed41b41c253099"}
Jul 17 18:11:36.231881 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:36.231847    2572 generic.go:356] "Generic (PLEG): container finished" podID="636f82e1-6c30-4536-af38-bd936909448d" containerID="802cf37df95e782e4ca577272e9d61dcf39ce2d0d0203488f76b4937599088da" exitCode=6
Jul 17 18:11:36.232334 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:36.231911    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt" event={"ID":"636f82e1-6c30-4536-af38-bd936909448d","Type":"ContainerDied","Data":"802cf37df95e782e4ca577272e9d61dcf39ce2d0d0203488f76b4937599088da"}
Jul 17 18:11:36.232334 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:36.232105    2572 scope.go:117] "RemoveContainer" containerID="802cf37df95e782e4ca577272e9d61dcf39ce2d0d0203488f76b4937599088da"
Jul 17 18:11:37.245560 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:37.245462    2572 generic.go:356] "Generic (PLEG): container finished" podID="636f82e1-6c30-4536-af38-bd936909448d" containerID="4d052a127ef635355ef001a75c9edbaadf4727947166d235968645a5b29eb98b" exitCode=6
Jul 17 18:11:37.245945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:37.245553    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt" event={"ID":"636f82e1-6c30-4536-af38-bd936909448d","Type":"ContainerDied","Data":"4d052a127ef635355ef001a75c9edbaadf4727947166d235968645a5b29eb98b"}
Jul 17 18:11:37.245945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:37.245649    2572 scope.go:117] "RemoveContainer" containerID="802cf37df95e782e4ca577272e9d61dcf39ce2d0d0203488f76b4937599088da"
Jul 17 18:11:37.245945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:37.245892    2572 scope.go:117] "RemoveContainer" containerID="4d052a127ef635355ef001a75c9edbaadf4727947166d235968645a5b29eb98b"
Jul 17 18:11:37.246129 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:37.246106    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt_odh-ai-gateway-infra(636f82e1-6c30-4536-af38-bd936909448d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt" podUID="636f82e1-6c30-4536-af38-bd936909448d"
Jul 17 18:11:38.253447 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:38.253418    2572 scope.go:117] "RemoveContainer" containerID="4d052a127ef635355ef001a75c9edbaadf4727947166d235968645a5b29eb98b"
Jul 17 18:11:38.253860 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:38.253660    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt_odh-ai-gateway-infra(636f82e1-6c30-4536-af38-bd936909448d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt" podUID="636f82e1-6c30-4536-af38-bd936909448d"
Jul 17 18:11:45.509932 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:45.509844    2572 scope.go:117] "RemoveContainer" containerID="fe65a97170bcd0d79dd712289d8e53749d49554334c014debdbb6e27c289cd58"
Jul 17 18:11:46.295767 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:46.295729    2572 generic.go:356] "Generic (PLEG): container finished" podID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerID="6f5c69d89c4266b9c881f94ec7498397cd3d0632b1dd020ddf24bb7fbcfd0173" exitCode=6
Jul 17 18:11:46.295934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:46.295805    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98" event={"ID":"b14b97b0-dec3-48d9-b61d-18236b385bdc","Type":"ContainerDied","Data":"6f5c69d89c4266b9c881f94ec7498397cd3d0632b1dd020ddf24bb7fbcfd0173"}
Jul 17 18:11:46.295934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:46.295876    2572 scope.go:117] "RemoveContainer" containerID="fe65a97170bcd0d79dd712289d8e53749d49554334c014debdbb6e27c289cd58"
Jul 17 18:11:46.296202 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:46.296184    2572 scope.go:117] "RemoveContainer" containerID="6f5c69d89c4266b9c881f94ec7498397cd3d0632b1dd020ddf24bb7fbcfd0173"
Jul 17 18:11:46.296439 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:46.296419    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-l9d98_odh-ai-gateway-infra(b14b97b0-dec3-48d9-b61d-18236b385bdc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc"
Jul 17 18:11:47.324171 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.324130    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"]
Jul 17 18:11:47.462951 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.462924    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"
Jul 17 18:11:47.507039 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.506997    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14b97b0-dec3-48d9-b61d-18236b385bdc-maas-api-service-ca\") pod \"b14b97b0-dec3-48d9-b61d-18236b385bdc\" (UID: \"b14b97b0-dec3-48d9-b61d-18236b385bdc\") "
Jul 17 18:11:47.507366 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.507343    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b14b97b0-dec3-48d9-b61d-18236b385bdc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b14b97b0-dec3-48d9-b61d-18236b385bdc" (UID: "b14b97b0-dec3-48d9-b61d-18236b385bdc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:11:47.593000 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.592905    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"]
Jul 17 18:11:47.593330 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.593314    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerName="revoke-keys"
Jul 17 18:11:47.593373 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.593332    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerName="revoke-keys"
Jul 17 18:11:47.593373 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.593343    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerName="revoke-keys"
Jul 17 18:11:47.593373 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.593348    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerName="revoke-keys"
Jul 17 18:11:47.593479 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.593408    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerName="revoke-keys"
Jul 17 18:11:47.593479 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.593416    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerName="revoke-keys"
Jul 17 18:11:47.596401 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.596380    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.599159 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.599134    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c50cdb\"/\"openshift-service-ca.crt\""
Jul 17 18:11:47.599598 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.599576    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c50cdb\"/\"test-model-c50cdb-kserve-self-signed-certs\""
Jul 17 18:11:47.600008 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.599990    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c50cdb\"/\"default-dockercfg-nclzr\""
Jul 17 18:11:47.600008 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.599999    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c50cdb\"/\"kube-root-ca.crt\""
Jul 17 18:11:47.607417 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.607387    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"]
Jul 17 18:11:47.607616 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.607590    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bea691-22d2-40f0-837d-da02483ffa87-tls-certs\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.607685 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.607633    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-home\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.607817 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.607695    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-model-cache\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.607817 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.607711    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-tmp-dir\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.607817 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.607728    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82vq4\" (UniqueName: \"kubernetes.io/projected/a4bea691-22d2-40f0-837d-da02483ffa87-kube-api-access-82vq4\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.607817 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.607747    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-dshm\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.607817 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.607798    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14b97b0-dec3-48d9-b61d-18236b385bdc-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:11:47.708159 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.708113    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-dshm\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.708353 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.708206    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bea691-22d2-40f0-837d-da02483ffa87-tls-certs\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.708353 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.708236    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-home\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.708353 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.708283    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-model-cache\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.708544 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.708411    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-tmp-dir\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.708544 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.708453    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82vq4\" (UniqueName: \"kubernetes.io/projected/a4bea691-22d2-40f0-837d-da02483ffa87-kube-api-access-82vq4\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.708693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.708670    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-home\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.708751 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.708676    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-model-cache\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.708751 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.708727    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-tmp-dir\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.710576 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.710558    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-dshm\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.710852 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.710836    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bea691-22d2-40f0-837d-da02483ffa87-tls-certs\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.726782 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.726743    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82vq4\" (UniqueName: \"kubernetes.io/projected/a4bea691-22d2-40f0-837d-da02483ffa87-kube-api-access-82vq4\") pod \"test-model-c50cdb-kserve-66b77fb4c5-hmzkc\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") " pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:47.907755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:47.907644    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:48.054199 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.054166    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"]
Jul 17 18:11:48.055837 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:11:48.055812    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4bea691_22d2_40f0_837d_da02483ffa87.slice/crio-6bc50e4a9a985a1c57753e492988b01698c12870073ca01825b867d4b23bbcd1 WatchSource:0}: Error finding container 6bc50e4a9a985a1c57753e492988b01698c12870073ca01825b867d4b23bbcd1: Status 404 returned error can't find the container with id 6bc50e4a9a985a1c57753e492988b01698c12870073ca01825b867d4b23bbcd1
Jul 17 18:11:48.309475 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.309432    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc" event={"ID":"a4bea691-22d2-40f0-837d-da02483ffa87","Type":"ContainerStarted","Data":"416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce"}
Jul 17 18:11:48.309475 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.309500    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc" event={"ID":"a4bea691-22d2-40f0-837d-da02483ffa87","Type":"ContainerStarted","Data":"6bc50e4a9a985a1c57753e492988b01698c12870073ca01825b867d4b23bbcd1"}
Jul 17 18:11:48.310905 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.310881    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"
Jul 17 18:11:48.310905 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.310891    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98" event={"ID":"b14b97b0-dec3-48d9-b61d-18236b385bdc","Type":"ContainerDied","Data":"fe72e72f09d4bbe1b8ac23a39463dd6c9f2a4bc170a7676ec7a3b8f26abd6634"}
Jul 17 18:11:48.311066 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.310943    2572 scope.go:117] "RemoveContainer" containerID="6f5c69d89c4266b9c881f94ec7498397cd3d0632b1dd020ddf24bb7fbcfd0173"
Jul 17 18:11:48.333016 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.332956    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc" podStartSLOduration=1.3329365659999999 podStartE2EDuration="1.332936566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:11:48.330637087 +0000 UTC m=+2880.335992958" watchObservedRunningTime="2026-07-17 18:11:48.332936566 +0000 UTC m=+2880.338292486"
Jul 17 18:11:48.357268 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.357224    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"]
Jul 17 18:11:48.359502 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.359468    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-l9d98"]
Jul 17 18:11:48.515955 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.515923    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" path="/var/lib/kubelet/pods/b14b97b0-dec3-48d9-b61d-18236b385bdc/volumes"
Jul 17 18:11:48.571139 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.571054    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"]
Jul 17 18:11:48.571411 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.571398    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerName="revoke-keys"
Jul 17 18:11:48.571458 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.571413    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerName="revoke-keys"
Jul 17 18:11:48.571535 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.571524    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14b97b0-dec3-48d9-b61d-18236b385bdc" containerName="revoke-keys"
Jul 17 18:11:48.575992 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.575972    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"
Jul 17 18:11:48.582904 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.582862    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"]
Jul 17 18:11:48.614677 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.614639    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e219a0-d306-4ad3-95d3-769d38b66f6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl\" (UID: \"a0e219a0-d306-4ad3-95d3-769d38b66f6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"
Jul 17 18:11:48.715151 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.715118    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e219a0-d306-4ad3-95d3-769d38b66f6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl\" (UID: \"a0e219a0-d306-4ad3-95d3-769d38b66f6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"
Jul 17 18:11:48.715739 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.715719    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e219a0-d306-4ad3-95d3-769d38b66f6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl\" (UID: \"a0e219a0-d306-4ad3-95d3-769d38b66f6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"
Jul 17 18:11:48.888897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:48.888807    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"
Jul 17 18:11:49.025625 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:49.025601    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"]
Jul 17 18:11:49.027612 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:11:49.027572    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e219a0_d306_4ad3_95d3_769d38b66f6d.slice/crio-0a8dfdacd8b63e2476728a8f32486878039647ef9ccb3eb080c3abec5104a8b4 WatchSource:0}: Error finding container 0a8dfdacd8b63e2476728a8f32486878039647ef9ccb3eb080c3abec5104a8b4: Status 404 returned error can't find the container with id 0a8dfdacd8b63e2476728a8f32486878039647ef9ccb3eb080c3abec5104a8b4
Jul 17 18:11:49.317078 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:49.317040    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl" event={"ID":"a0e219a0-d306-4ad3-95d3-769d38b66f6d","Type":"ContainerStarted","Data":"0a8dfdacd8b63e2476728a8f32486878039647ef9ccb3eb080c3abec5104a8b4"}
Jul 17 18:11:50.324397 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:50.324364    2572 generic.go:356] "Generic (PLEG): container finished" podID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerID="44b033633c57525f7e6fca6f2e187d0c8097fc1831780156d3d637b45357ef8a" exitCode=6
Jul 17 18:11:50.324848 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:50.324456    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl" event={"ID":"a0e219a0-d306-4ad3-95d3-769d38b66f6d","Type":"ContainerDied","Data":"44b033633c57525f7e6fca6f2e187d0c8097fc1831780156d3d637b45357ef8a"}
Jul 17 18:11:50.324848 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:50.324640    2572 scope.go:117] "RemoveContainer" containerID="44b033633c57525f7e6fca6f2e187d0c8097fc1831780156d3d637b45357ef8a"
Jul 17 18:11:51.332384 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:51.332346    2572 generic.go:356] "Generic (PLEG): container finished" podID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerID="9edcf1966eec285dc5a12ce9b35b4478d353dc8636b022dee28bc31249779071" exitCode=6
Jul 17 18:11:51.332863 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:51.332433    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl" event={"ID":"a0e219a0-d306-4ad3-95d3-769d38b66f6d","Type":"ContainerDied","Data":"9edcf1966eec285dc5a12ce9b35b4478d353dc8636b022dee28bc31249779071"}
Jul 17 18:11:51.332863 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:51.332562    2572 scope.go:117] "RemoveContainer" containerID="44b033633c57525f7e6fca6f2e187d0c8097fc1831780156d3d637b45357ef8a"
Jul 17 18:11:51.332863 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:51.332658    2572 scope.go:117] "RemoveContainer" containerID="9edcf1966eec285dc5a12ce9b35b4478d353dc8636b022dee28bc31249779071"
Jul 17 18:11:51.332863 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:51.332849    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl_odh-ai-gateway-infra(a0e219a0-d306-4ad3-95d3-769d38b66f6d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d"
Jul 17 18:11:52.340182 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:52.340150    2572 scope.go:117] "RemoveContainer" containerID="9edcf1966eec285dc5a12ce9b35b4478d353dc8636b022dee28bc31249779071"
Jul 17 18:11:52.340595 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:52.340352    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl_odh-ai-gateway-infra(a0e219a0-d306-4ad3-95d3-769d38b66f6d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d"
Jul 17 18:11:53.510567 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:53.510534    2572 scope.go:117] "RemoveContainer" containerID="4d052a127ef635355ef001a75c9edbaadf4727947166d235968645a5b29eb98b"
Jul 17 18:11:54.359351 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:54.359315    2572 generic.go:356] "Generic (PLEG): container finished" podID="636f82e1-6c30-4536-af38-bd936909448d" containerID="23214b9e769b921c5a319ef1eecff5cae6a15a6aaad6ce1c227f9c1416e12cdf" exitCode=6
Jul 17 18:11:54.359587 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:54.359387    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt" event={"ID":"636f82e1-6c30-4536-af38-bd936909448d","Type":"ContainerDied","Data":"23214b9e769b921c5a319ef1eecff5cae6a15a6aaad6ce1c227f9c1416e12cdf"}
Jul 17 18:11:54.359587 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:54.359511    2572 scope.go:117] "RemoveContainer" containerID="4d052a127ef635355ef001a75c9edbaadf4727947166d235968645a5b29eb98b"
Jul 17 18:11:54.359809 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:54.359786    2572 scope.go:117] "RemoveContainer" containerID="23214b9e769b921c5a319ef1eecff5cae6a15a6aaad6ce1c227f9c1416e12cdf"
Jul 17 18:11:54.359999 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:54.359979    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt_odh-ai-gateway-infra(636f82e1-6c30-4536-af38-bd936909448d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt" podUID="636f82e1-6c30-4536-af38-bd936909448d"
Jul 17 18:11:55.386255 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:55.386216    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"]
Jul 17 18:11:55.527083 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:55.527047    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"
Jul 17 18:11:55.565678 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:55.565643    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636f82e1-6c30-4536-af38-bd936909448d-maas-api-service-ca\") pod \"636f82e1-6c30-4536-af38-bd936909448d\" (UID: \"636f82e1-6c30-4536-af38-bd936909448d\") "
Jul 17 18:11:55.565992 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:55.565969    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636f82e1-6c30-4536-af38-bd936909448d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "636f82e1-6c30-4536-af38-bd936909448d" (UID: "636f82e1-6c30-4536-af38-bd936909448d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:11:55.666474 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:55.666375    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636f82e1-6c30-4536-af38-bd936909448d-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:11:56.373660 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.373624    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"
Jul 17 18:11:56.373660 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.373653    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt" event={"ID":"636f82e1-6c30-4536-af38-bd936909448d","Type":"ContainerDied","Data":"8ceca2cc3b5c7c29a2ec382af163e9a69513392b28af38f226ed41b41c253099"}
Jul 17 18:11:56.373944 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.373718    2572 scope.go:117] "RemoveContainer" containerID="23214b9e769b921c5a319ef1eecff5cae6a15a6aaad6ce1c227f9c1416e12cdf"
Jul 17 18:11:56.398556 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.398516    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"]
Jul 17 18:11:56.402319 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.402285    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lvkrt"]
Jul 17 18:11:56.515334 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.515297    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="636f82e1-6c30-4536-af38-bd936909448d" path="/var/lib/kubelet/pods/636f82e1-6c30-4536-af38-bd936909448d/volumes"
Jul 17 18:11:56.729015 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.728928    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"]
Jul 17 18:11:56.729427 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.729412    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="636f82e1-6c30-4536-af38-bd936909448d" containerName="revoke-keys"
Jul 17 18:11:56.729474 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.729431    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="636f82e1-6c30-4536-af38-bd936909448d" containerName="revoke-keys"
Jul 17 18:11:56.729474 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.729465    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="636f82e1-6c30-4536-af38-bd936909448d" containerName="revoke-keys"
Jul 17 18:11:56.729566 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.729473    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="636f82e1-6c30-4536-af38-bd936909448d" containerName="revoke-keys"
Jul 17 18:11:56.729599 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.729572    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="636f82e1-6c30-4536-af38-bd936909448d" containerName="revoke-keys"
Jul 17 18:11:56.729599 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.729582    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="636f82e1-6c30-4536-af38-bd936909448d" containerName="revoke-keys"
Jul 17 18:11:56.729599 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.729594    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="636f82e1-6c30-4536-af38-bd936909448d" containerName="revoke-keys"
Jul 17 18:11:56.734485 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.734457    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"
Jul 17 18:11:56.739005 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.738975    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"]
Jul 17 18:11:56.775716 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.775672    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e72446-bbd3-442b-a020-f4a4cc52c6f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-dnncm\" (UID: \"03e72446-bbd3-442b-a020-f4a4cc52c6f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"
Jul 17 18:11:56.876332 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.876287    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e72446-bbd3-442b-a020-f4a4cc52c6f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-dnncm\" (UID: \"03e72446-bbd3-442b-a020-f4a4cc52c6f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"
Jul 17 18:11:56.876974 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:56.876954    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e72446-bbd3-442b-a020-f4a4cc52c6f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-dnncm\" (UID: \"03e72446-bbd3-442b-a020-f4a4cc52c6f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"
Jul 17 18:11:57.046415 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:57.046311    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"
Jul 17 18:11:57.187794 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:57.187766    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"]
Jul 17 18:11:57.189660 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:11:57.189620    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e72446_bbd3_442b_a020_f4a4cc52c6f3.slice/crio-0db6e0c19f2ddc4c1ea83eb6dd95baeb0d79aa385c01db48b4b3a6b9691b7801 WatchSource:0}: Error finding container 0db6e0c19f2ddc4c1ea83eb6dd95baeb0d79aa385c01db48b4b3a6b9691b7801: Status 404 returned error can't find the container with id 0db6e0c19f2ddc4c1ea83eb6dd95baeb0d79aa385c01db48b4b3a6b9691b7801
Jul 17 18:11:57.381528 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:57.381476    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm" event={"ID":"03e72446-bbd3-442b-a020-f4a4cc52c6f3","Type":"ContainerStarted","Data":"0db6e0c19f2ddc4c1ea83eb6dd95baeb0d79aa385c01db48b4b3a6b9691b7801"}
Jul 17 18:11:57.908648 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:57.908610    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:57.909035 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:57.908670    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:57.921078 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:57.921048    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:58.388720 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:58.388683    2572 generic.go:356] "Generic (PLEG): container finished" podID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerID="20d39f9a4551d6904562ea20768a3c97bb7cf1443d25fad37f96cfb170c2c0dd" exitCode=6
Jul 17 18:11:58.388920 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:58.388720    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm" event={"ID":"03e72446-bbd3-442b-a020-f4a4cc52c6f3","Type":"ContainerDied","Data":"20d39f9a4551d6904562ea20768a3c97bb7cf1443d25fad37f96cfb170c2c0dd"}
Jul 17 18:11:58.388983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:58.388936    2572 scope.go:117] "RemoveContainer" containerID="20d39f9a4551d6904562ea20768a3c97bb7cf1443d25fad37f96cfb170c2c0dd"
Jul 17 18:11:58.400955 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:58.400929    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:11:59.396380 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:59.396337    2572 generic.go:356] "Generic (PLEG): container finished" podID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerID="8363e45081ca9506bc0ea4dc2ca0941199c0cc4d6d3bf76873c5ea375de60325" exitCode=6
Jul 17 18:11:59.396868 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:59.396418    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm" event={"ID":"03e72446-bbd3-442b-a020-f4a4cc52c6f3","Type":"ContainerDied","Data":"8363e45081ca9506bc0ea4dc2ca0941199c0cc4d6d3bf76873c5ea375de60325"}
Jul 17 18:11:59.396868 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:59.396527    2572 scope.go:117] "RemoveContainer" containerID="20d39f9a4551d6904562ea20768a3c97bb7cf1443d25fad37f96cfb170c2c0dd"
Jul 17 18:11:59.396868 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:11:59.396697    2572 scope.go:117] "RemoveContainer" containerID="8363e45081ca9506bc0ea4dc2ca0941199c0cc4d6d3bf76873c5ea375de60325"
Jul 17 18:11:59.397101 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:11:59.396960    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-dnncm_odh-ai-gateway-infra(03e72446-bbd3-442b-a020-f4a4cc52c6f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3"
Jul 17 18:12:00.404337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:00.404310    2572 scope.go:117] "RemoveContainer" containerID="8363e45081ca9506bc0ea4dc2ca0941199c0cc4d6d3bf76873c5ea375de60325"
Jul 17 18:12:00.404844 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:00.404615    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-dnncm_odh-ai-gateway-infra(03e72446-bbd3-442b-a020-f4a4cc52c6f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3"
Jul 17 18:12:07.509789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.509756    2572 scope.go:117] "RemoveContainer" containerID="9edcf1966eec285dc5a12ce9b35b4478d353dc8636b022dee28bc31249779071"
Jul 17 18:12:07.660440 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.649480    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"]
Jul 17 18:12:07.766081 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.765992    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"]
Jul 17 18:12:07.766374 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.766362    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="636f82e1-6c30-4536-af38-bd936909448d" containerName="revoke-keys"
Jul 17 18:12:07.766429 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.766376    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="636f82e1-6c30-4536-af38-bd936909448d" containerName="revoke-keys"
Jul 17 18:12:07.770822 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.770802    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"
Jul 17 18:12:07.777316 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.777281    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"]
Jul 17 18:12:07.864772 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.864723    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f6ed61-e321-4a30-ae79-4e9a9248e5cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d\" (UID: \"96f6ed61-e321-4a30-ae79-4e9a9248e5cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"
Jul 17 18:12:07.965374 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.965333    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f6ed61-e321-4a30-ae79-4e9a9248e5cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d\" (UID: \"96f6ed61-e321-4a30-ae79-4e9a9248e5cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"
Jul 17 18:12:07.965991 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:07.965968    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f6ed61-e321-4a30-ae79-4e9a9248e5cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d\" (UID: \"96f6ed61-e321-4a30-ae79-4e9a9248e5cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"
Jul 17 18:12:08.084435 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:08.084397    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"
Jul 17 18:12:08.227176 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:08.227151    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"]
Jul 17 18:12:08.229891 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:12:08.229848    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f6ed61_e321_4a30_ae79_4e9a9248e5cf.slice/crio-c038bfcc246ecd13d87d58fd5381785d5551b1f0e7eba2c87f8aa2985acb4467 WatchSource:0}: Error finding container c038bfcc246ecd13d87d58fd5381785d5551b1f0e7eba2c87f8aa2985acb4467: Status 404 returned error can't find the container with id c038bfcc246ecd13d87d58fd5381785d5551b1f0e7eba2c87f8aa2985acb4467
Jul 17 18:12:08.452460 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:08.452421    2572 generic.go:356] "Generic (PLEG): container finished" podID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerID="23cddd55470e903e95d6d8bdb2bd27861a580a0d232630e110000d3119c79ab4" exitCode=6
Jul 17 18:12:08.452684 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:08.452519    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl" event={"ID":"a0e219a0-d306-4ad3-95d3-769d38b66f6d","Type":"ContainerDied","Data":"23cddd55470e903e95d6d8bdb2bd27861a580a0d232630e110000d3119c79ab4"}
Jul 17 18:12:08.452684 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:08.452592    2572 scope.go:117] "RemoveContainer" containerID="9edcf1966eec285dc5a12ce9b35b4478d353dc8636b022dee28bc31249779071"
Jul 17 18:12:08.452924 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:08.452907    2572 scope.go:117] "RemoveContainer" containerID="23cddd55470e903e95d6d8bdb2bd27861a580a0d232630e110000d3119c79ab4"
Jul 17 18:12:08.453186 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:08.453162    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl_odh-ai-gateway-infra(a0e219a0-d306-4ad3-95d3-769d38b66f6d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d"
Jul 17 18:12:08.453980 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:08.453961    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d" event={"ID":"96f6ed61-e321-4a30-ae79-4e9a9248e5cf","Type":"ContainerStarted","Data":"c038bfcc246ecd13d87d58fd5381785d5551b1f0e7eba2c87f8aa2985acb4467"}
Jul 17 18:12:09.462789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:09.462693    2572 generic.go:356] "Generic (PLEG): container finished" podID="96f6ed61-e321-4a30-ae79-4e9a9248e5cf" containerID="2b2350d31ad1aed90cc06e5fd7b38f1fcd90a9af6ef40830111015ed96a78507" exitCode=0
Jul 17 18:12:09.463181 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:09.462783    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d" event={"ID":"96f6ed61-e321-4a30-ae79-4e9a9248e5cf","Type":"ContainerDied","Data":"2b2350d31ad1aed90cc06e5fd7b38f1fcd90a9af6ef40830111015ed96a78507"}
Jul 17 18:12:09.501923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:09.501879    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"]
Jul 17 18:12:09.640724 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:09.640693    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"
Jul 17 18:12:09.678631 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:09.678587    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e219a0-d306-4ad3-95d3-769d38b66f6d-maas-api-service-ca\") pod \"a0e219a0-d306-4ad3-95d3-769d38b66f6d\" (UID: \"a0e219a0-d306-4ad3-95d3-769d38b66f6d\") "
Jul 17 18:12:09.678992 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:09.678970    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0e219a0-d306-4ad3-95d3-769d38b66f6d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a0e219a0-d306-4ad3-95d3-769d38b66f6d" (UID: "a0e219a0-d306-4ad3-95d3-769d38b66f6d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:12:09.780012 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:09.779915    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e219a0-d306-4ad3-95d3-769d38b66f6d-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:10.469847 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.469806    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"
Jul 17 18:12:10.469847 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.469835    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl" event={"ID":"a0e219a0-d306-4ad3-95d3-769d38b66f6d","Type":"ContainerDied","Data":"0a8dfdacd8b63e2476728a8f32486878039647ef9ccb3eb080c3abec5104a8b4"}
Jul 17 18:12:10.470431 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.469941    2572 scope.go:117] "RemoveContainer" containerID="23cddd55470e903e95d6d8bdb2bd27861a580a0d232630e110000d3119c79ab4"
Jul 17 18:12:10.494998 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.494958    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"]
Jul 17 18:12:10.499545 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.499513    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-q6mwl"]
Jul 17 18:12:10.516510 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.516467    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" path="/var/lib/kubelet/pods/a0e219a0-d306-4ad3-95d3-769d38b66f6d/volumes"
Jul 17 18:12:10.610270 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.610238    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"
Jul 17 18:12:10.687509 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.687455    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f6ed61-e321-4a30-ae79-4e9a9248e5cf-maas-api-service-ca\") pod \"96f6ed61-e321-4a30-ae79-4e9a9248e5cf\" (UID: \"96f6ed61-e321-4a30-ae79-4e9a9248e5cf\") "
Jul 17 18:12:10.687892 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.687869    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96f6ed61-e321-4a30-ae79-4e9a9248e5cf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96f6ed61-e321-4a30-ae79-4e9a9248e5cf" (UID: "96f6ed61-e321-4a30-ae79-4e9a9248e5cf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:12:10.788868 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:10.788770    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f6ed61-e321-4a30-ae79-4e9a9248e5cf-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:11.477153 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:11.477118    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d"
Jul 17 18:12:11.477153 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:11.477143    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d" event={"ID":"96f6ed61-e321-4a30-ae79-4e9a9248e5cf","Type":"ContainerDied","Data":"c038bfcc246ecd13d87d58fd5381785d5551b1f0e7eba2c87f8aa2985acb4467"}
Jul 17 18:12:11.477646 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:11.477197    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c038bfcc246ecd13d87d58fd5381785d5551b1f0e7eba2c87f8aa2985acb4467"
Jul 17 18:12:11.510336 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:11.510297    2572 scope.go:117] "RemoveContainer" containerID="8363e45081ca9506bc0ea4dc2ca0941199c0cc4d6d3bf76873c5ea375de60325"
Jul 17 18:12:12.484593 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:12.484480    2572 generic.go:356] "Generic (PLEG): container finished" podID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerID="ee690893c0d58ac129c4f541a383e70143305eb0314e3052df3907ac742da7ee" exitCode=6
Jul 17 18:12:12.484593 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:12.484564    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm" event={"ID":"03e72446-bbd3-442b-a020-f4a4cc52c6f3","Type":"ContainerDied","Data":"ee690893c0d58ac129c4f541a383e70143305eb0314e3052df3907ac742da7ee"}
Jul 17 18:12:12.485135 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:12.484629    2572 scope.go:117] "RemoveContainer" containerID="8363e45081ca9506bc0ea4dc2ca0941199c0cc4d6d3bf76873c5ea375de60325"
Jul 17 18:12:12.485135 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:12.485012    2572 scope.go:117] "RemoveContainer" containerID="ee690893c0d58ac129c4f541a383e70143305eb0314e3052df3907ac742da7ee"
Jul 17 18:12:12.485281 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:12.485258    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-dnncm_odh-ai-gateway-infra(03e72446-bbd3-442b-a020-f4a4cc52c6f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3"
Jul 17 18:12:13.511386 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:13.511348    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"]
Jul 17 18:12:13.650922 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:13.650894    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"
Jul 17 18:12:13.715600 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:13.715563    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e72446-bbd3-442b-a020-f4a4cc52c6f3-maas-api-service-ca\") pod \"03e72446-bbd3-442b-a020-f4a4cc52c6f3\" (UID: \"03e72446-bbd3-442b-a020-f4a4cc52c6f3\") "
Jul 17 18:12:13.715944 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:13.715920    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03e72446-bbd3-442b-a020-f4a4cc52c6f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "03e72446-bbd3-442b-a020-f4a4cc52c6f3" (UID: "03e72446-bbd3-442b-a020-f4a4cc52c6f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:12:13.817035 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:13.816996    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03e72446-bbd3-442b-a020-f4a4cc52c6f3-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:14.499632 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:14.499600    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"
Jul 17 18:12:14.499825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:14.499639    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm" event={"ID":"03e72446-bbd3-442b-a020-f4a4cc52c6f3","Type":"ContainerDied","Data":"0db6e0c19f2ddc4c1ea83eb6dd95baeb0d79aa385c01db48b4b3a6b9691b7801"}
Jul 17 18:12:14.499825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:14.499707    2572 scope.go:117] "RemoveContainer" containerID="ee690893c0d58ac129c4f541a383e70143305eb0314e3052df3907ac742da7ee"
Jul 17 18:12:14.527852 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:14.527807    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"]
Jul 17 18:12:14.537283 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:14.537246    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dnncm"]
Jul 17 18:12:16.517035 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:16.516998    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" path="/var/lib/kubelet/pods/03e72446-bbd3-442b-a020-f4a4cc52c6f3/volumes"
Jul 17 18:12:20.032721 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.032686    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"]
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033026    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f6ed61-e321-4a30-ae79-4e9a9248e5cf" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033037    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f6ed61-e321-4a30-ae79-4e9a9248e5cf" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033045    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033050    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033057    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033063    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033068    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033073    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033080    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033085    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033092    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerName="revoke-keys"
Jul 17 18:12:20.033118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033097    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerName="revoke-keys"
Jul 17 18:12:20.033480 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033152    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="96f6ed61-e321-4a30-ae79-4e9a9248e5cf" containerName="revoke-keys"
Jul 17 18:12:20.033480 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033161    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerName="revoke-keys"
Jul 17 18:12:20.033480 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033167    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerName="revoke-keys"
Jul 17 18:12:20.033480 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033174    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerName="revoke-keys"
Jul 17 18:12:20.033480 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033180    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerName="revoke-keys"
Jul 17 18:12:20.033480 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.033186    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="03e72446-bbd3-442b-a020-f4a4cc52c6f3" containerName="revoke-keys"
Jul 17 18:12:20.038063 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.038040    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"
Jul 17 18:12:20.042791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.042765    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:12:20.042945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.042765    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:12:20.045186 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.045156    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"]
Jul 17 18:12:20.074318 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.074264    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20333f6f-4c9e-4dc0-989e-52f471fa9559-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv\" (UID: \"20333f6f-4c9e-4dc0-989e-52f471fa9559\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"
Jul 17 18:12:20.175628 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.175583    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20333f6f-4c9e-4dc0-989e-52f471fa9559-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv\" (UID: \"20333f6f-4c9e-4dc0-989e-52f471fa9559\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"
Jul 17 18:12:20.176254 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.176233    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20333f6f-4c9e-4dc0-989e-52f471fa9559-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv\" (UID: \"20333f6f-4c9e-4dc0-989e-52f471fa9559\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"
Jul 17 18:12:20.349781 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.349735    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"
Jul 17 18:12:20.486514 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.486462    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"]
Jul 17 18:12:20.488940 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:12:20.488910    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20333f6f_4c9e_4dc0_989e_52f471fa9559.slice/crio-8bfb26b917a4ddd2962ac9050b9b521f204da9172ae2ffb10a3791992213f09d WatchSource:0}: Error finding container 8bfb26b917a4ddd2962ac9050b9b521f204da9172ae2ffb10a3791992213f09d: Status 404 returned error can't find the container with id 8bfb26b917a4ddd2962ac9050b9b521f204da9172ae2ffb10a3791992213f09d
Jul 17 18:12:20.537198 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:20.537157    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv" event={"ID":"20333f6f-4c9e-4dc0-989e-52f471fa9559","Type":"ContainerStarted","Data":"8bfb26b917a4ddd2962ac9050b9b521f204da9172ae2ffb10a3791992213f09d"}
Jul 17 18:12:21.543671 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:21.543567    2572 generic.go:356] "Generic (PLEG): container finished" podID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerID="a02cffe3218b9d6251fb91f7e3fd56fa1589ed3be7b200d98b3058b6aa1fe964" exitCode=6
Jul 17 18:12:21.543671 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:21.543654    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv" event={"ID":"20333f6f-4c9e-4dc0-989e-52f471fa9559","Type":"ContainerDied","Data":"a02cffe3218b9d6251fb91f7e3fd56fa1589ed3be7b200d98b3058b6aa1fe964"}
Jul 17 18:12:21.544104 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:21.543861    2572 scope.go:117] "RemoveContainer" containerID="a02cffe3218b9d6251fb91f7e3fd56fa1589ed3be7b200d98b3058b6aa1fe964"
Jul 17 18:12:22.551275 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:22.551181    2572 generic.go:356] "Generic (PLEG): container finished" podID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerID="062da7307cf64370950b21252ecf7eb2ce097d0ccb171af308ea683a1c70e1b9" exitCode=6
Jul 17 18:12:22.551275 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:22.551256    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv" event={"ID":"20333f6f-4c9e-4dc0-989e-52f471fa9559","Type":"ContainerDied","Data":"062da7307cf64370950b21252ecf7eb2ce097d0ccb171af308ea683a1c70e1b9"}
Jul 17 18:12:22.551805 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:22.551305    2572 scope.go:117] "RemoveContainer" containerID="a02cffe3218b9d6251fb91f7e3fd56fa1589ed3be7b200d98b3058b6aa1fe964"
Jul 17 18:12:22.551805 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:22.551527    2572 scope.go:117] "RemoveContainer" containerID="062da7307cf64370950b21252ecf7eb2ce097d0ccb171af308ea683a1c70e1b9"
Jul 17 18:12:22.551805 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:22.551787    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv_odh-ai-gateway-infra(20333f6f-4c9e-4dc0-989e-52f471fa9559)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559"
Jul 17 18:12:22.680375 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:22.680325    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc" podUID="a4bea691-22d2-40f0-837d-da02483ffa87" containerName="main" containerID="cri-o://416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce" gracePeriod=45
Jul 17 18:12:22.945302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:22.945276    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:12:23.000514 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.000460    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-dshm\") pod \"a4bea691-22d2-40f0-837d-da02483ffa87\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") "
Jul 17 18:12:23.000726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.000532    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-model-cache\") pod \"a4bea691-22d2-40f0-837d-da02483ffa87\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") "
Jul 17 18:12:23.000726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.000566    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bea691-22d2-40f0-837d-da02483ffa87-tls-certs\") pod \"a4bea691-22d2-40f0-837d-da02483ffa87\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") "
Jul 17 18:12:23.000726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.000604    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-home\") pod \"a4bea691-22d2-40f0-837d-da02483ffa87\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") "
Jul 17 18:12:23.000726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.000712    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-tmp-dir\") pod \"a4bea691-22d2-40f0-837d-da02483ffa87\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") "
Jul 17 18:12:23.000973 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.000745    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82vq4\" (UniqueName: \"kubernetes.io/projected/a4bea691-22d2-40f0-837d-da02483ffa87-kube-api-access-82vq4\") pod \"a4bea691-22d2-40f0-837d-da02483ffa87\" (UID: \"a4bea691-22d2-40f0-837d-da02483ffa87\") "
Jul 17 18:12:23.000973 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.000858    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4bea691-22d2-40f0-837d-da02483ffa87" (UID: "a4bea691-22d2-40f0-837d-da02483ffa87"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:12:23.000973 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.000867    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-home" (OuterVolumeSpecName: "home") pod "a4bea691-22d2-40f0-837d-da02483ffa87" (UID: "a4bea691-22d2-40f0-837d-da02483ffa87"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:12:23.001138 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.000983    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4bea691-22d2-40f0-837d-da02483ffa87" (UID: "a4bea691-22d2-40f0-837d-da02483ffa87"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:12:23.001138 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.001113    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-home\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:23.001138 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.001136    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-tmp-dir\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:23.001298 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.001150    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-model-cache\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:23.002977 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.002940    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4bea691-22d2-40f0-837d-da02483ffa87-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4bea691-22d2-40f0-837d-da02483ffa87" (UID: "a4bea691-22d2-40f0-837d-da02483ffa87"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:12:23.002977 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.002955    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-dshm" (OuterVolumeSpecName: "dshm") pod "a4bea691-22d2-40f0-837d-da02483ffa87" (UID: "a4bea691-22d2-40f0-837d-da02483ffa87"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:12:23.003183 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.003003    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4bea691-22d2-40f0-837d-da02483ffa87-kube-api-access-82vq4" (OuterVolumeSpecName: "kube-api-access-82vq4") pod "a4bea691-22d2-40f0-837d-da02483ffa87" (UID: "a4bea691-22d2-40f0-837d-da02483ffa87"). InnerVolumeSpecName "kube-api-access-82vq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:12:23.101606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.101569    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bea691-22d2-40f0-837d-da02483ffa87-dshm\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:23.101606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.101597    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bea691-22d2-40f0-837d-da02483ffa87-tls-certs\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:23.101606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.101606    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82vq4\" (UniqueName: \"kubernetes.io/projected/a4bea691-22d2-40f0-837d-da02483ffa87-kube-api-access-82vq4\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:23.559447 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.559414    2572 scope.go:117] "RemoveContainer" containerID="062da7307cf64370950b21252ecf7eb2ce097d0ccb171af308ea683a1c70e1b9"
Jul 17 18:12:23.559931 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:23.559671    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv_odh-ai-gateway-infra(20333f6f-4c9e-4dc0-989e-52f471fa9559)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559"
Jul 17 18:12:23.560609 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.560582    2572 generic.go:356] "Generic (PLEG): container finished" podID="a4bea691-22d2-40f0-837d-da02483ffa87" containerID="416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce" exitCode=0
Jul 17 18:12:23.560705 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.560613    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"
Jul 17 18:12:23.560705 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.560619    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc" event={"ID":"a4bea691-22d2-40f0-837d-da02483ffa87","Type":"ContainerDied","Data":"416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce"}
Jul 17 18:12:23.560705 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.560648    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc" event={"ID":"a4bea691-22d2-40f0-837d-da02483ffa87","Type":"ContainerDied","Data":"6bc50e4a9a985a1c57753e492988b01698c12870073ca01825b867d4b23bbcd1"}
Jul 17 18:12:23.560705 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.560666    2572 scope.go:117] "RemoveContainer" containerID="416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce"
Jul 17 18:12:23.570227 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.570206    2572 scope.go:117] "RemoveContainer" containerID="416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce"
Jul 17 18:12:23.570613 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:23.570587    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce\": container with ID starting with 416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce not found: ID does not exist" containerID="416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce"
Jul 17 18:12:23.570682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.570626    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce"} err="failed to get container status \"416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce\": rpc error: code = NotFound desc = could not find container \"416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce\": container with ID starting with 416157f0297778021f695326f16456fa0571353fcd53f17f7d5b1c1e964bb2ce not found: ID does not exist"
Jul 17 18:12:23.589645 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.589606    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"]
Jul 17 18:12:23.596962 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:23.596934    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c50cdb/test-model-c50cdb-kserve-66b77fb4c5-hmzkc"]
Jul 17 18:12:24.516063 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:24.516030    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4bea691-22d2-40f0-837d-da02483ffa87" path="/var/lib/kubelet/pods/a4bea691-22d2-40f0-837d-da02483ffa87/volumes"
Jul 17 18:12:33.924838 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.924799    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"]
Jul 17 18:12:33.925270 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.925254    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4bea691-22d2-40f0-837d-da02483ffa87" containerName="main"
Jul 17 18:12:33.925318 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.925273    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4bea691-22d2-40f0-837d-da02483ffa87" containerName="main"
Jul 17 18:12:33.925318 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.925291    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerName="revoke-keys"
Jul 17 18:12:33.925318 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.925298    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerName="revoke-keys"
Jul 17 18:12:33.925421 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.925394    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4bea691-22d2-40f0-837d-da02483ffa87" containerName="main"
Jul 17 18:12:33.925421 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.925406    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e219a0-d306-4ad3-95d3-769d38b66f6d" containerName="revoke-keys"
Jul 17 18:12:33.928690 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.928673    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"
Jul 17 18:12:33.935168 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.935139    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"]
Jul 17 18:12:33.998472 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:33.998430    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8a0993-6dd7-445f-8bc1-3b914db314b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7\" (UID: \"dd8a0993-6dd7-445f-8bc1-3b914db314b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"
Jul 17 18:12:34.099459 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.099420    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8a0993-6dd7-445f-8bc1-3b914db314b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7\" (UID: \"dd8a0993-6dd7-445f-8bc1-3b914db314b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"
Jul 17 18:12:34.100134 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.100116    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8a0993-6dd7-445f-8bc1-3b914db314b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7\" (UID: \"dd8a0993-6dd7-445f-8bc1-3b914db314b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"
Jul 17 18:12:34.241444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.241353    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"
Jul 17 18:12:34.388460 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.388417    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"]
Jul 17 18:12:34.391639 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:12:34.391608    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8a0993_6dd7_445f_8bc1_3b914db314b3.slice/crio-a2e37db4f2492ce9d75dc7c1ead9378955fb8afb04c5e751a13e84919ec0e7ff WatchSource:0}: Error finding container a2e37db4f2492ce9d75dc7c1ead9378955fb8afb04c5e751a13e84919ec0e7ff: Status 404 returned error can't find the container with id a2e37db4f2492ce9d75dc7c1ead9378955fb8afb04c5e751a13e84919ec0e7ff
Jul 17 18:12:34.498832 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.498733    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"]
Jul 17 18:12:34.503201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.503171    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"
Jul 17 18:12:34.529234 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.529199    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"]
Jul 17 18:12:34.603715 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.603676    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1182526d-6b1a-4aab-879f-5a1e9ab65da2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p\" (UID: \"1182526d-6b1a-4aab-879f-5a1e9ab65da2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"
Jul 17 18:12:34.622322 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.622283    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7" event={"ID":"dd8a0993-6dd7-445f-8bc1-3b914db314b3","Type":"ContainerStarted","Data":"a2e37db4f2492ce9d75dc7c1ead9378955fb8afb04c5e751a13e84919ec0e7ff"}
Jul 17 18:12:34.704822 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.704776    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1182526d-6b1a-4aab-879f-5a1e9ab65da2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p\" (UID: \"1182526d-6b1a-4aab-879f-5a1e9ab65da2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"
Jul 17 18:12:34.705518 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.705470    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1182526d-6b1a-4aab-879f-5a1e9ab65da2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p\" (UID: \"1182526d-6b1a-4aab-879f-5a1e9ab65da2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"
Jul 17 18:12:34.828672 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.828632    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"
Jul 17 18:12:34.966934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:34.966906    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"]
Jul 17 18:12:34.968984 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:12:34.968952    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1182526d_6b1a_4aab_879f_5a1e9ab65da2.slice/crio-a7679f09412d93d5dc2d5fe23591d59bf355b1cafbe1850b10c268f7128f75d5 WatchSource:0}: Error finding container a7679f09412d93d5dc2d5fe23591d59bf355b1cafbe1850b10c268f7128f75d5: Status 404 returned error can't find the container with id a7679f09412d93d5dc2d5fe23591d59bf355b1cafbe1850b10c268f7128f75d5
Jul 17 18:12:35.632322 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:35.632292    2572 generic.go:356] "Generic (PLEG): container finished" podID="dd8a0993-6dd7-445f-8bc1-3b914db314b3" containerID="a510f78f04d119c208c656c2c29f3f1919b74f36bf8aa197342b7908026b49b9" exitCode=0
Jul 17 18:12:35.632458 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:35.632381    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7" event={"ID":"dd8a0993-6dd7-445f-8bc1-3b914db314b3","Type":"ContainerDied","Data":"a510f78f04d119c208c656c2c29f3f1919b74f36bf8aa197342b7908026b49b9"}
Jul 17 18:12:35.634092 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:35.634065    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" event={"ID":"1182526d-6b1a-4aab-879f-5a1e9ab65da2","Type":"ContainerStarted","Data":"de3d44f5eaa26f1b3f83fae9f500bc5e18ea75cb672b56e2d63581de848d89eb"}
Jul 17 18:12:35.634225 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:35.634123    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" event={"ID":"1182526d-6b1a-4aab-879f-5a1e9ab65da2","Type":"ContainerStarted","Data":"a7679f09412d93d5dc2d5fe23591d59bf355b1cafbe1850b10c268f7128f75d5"}
Jul 17 18:12:35.669911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:35.669850    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" podStartSLOduration=1.6698293290000001 podStartE2EDuration="1.669829329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:12:35.667539578 +0000 UTC m=+2927.672895439" watchObservedRunningTime="2026-07-17 18:12:35.669829329 +0000 UTC m=+2927.675185183"
Jul 17 18:12:36.642091 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:36.642056    2572 generic.go:356] "Generic (PLEG): container finished" podID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerID="de3d44f5eaa26f1b3f83fae9f500bc5e18ea75cb672b56e2d63581de848d89eb" exitCode=6
Jul 17 18:12:36.642543 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:36.642124    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" event={"ID":"1182526d-6b1a-4aab-879f-5a1e9ab65da2","Type":"ContainerDied","Data":"de3d44f5eaa26f1b3f83fae9f500bc5e18ea75cb672b56e2d63581de848d89eb"}
Jul 17 18:12:36.642543 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:36.642370    2572 scope.go:117] "RemoveContainer" containerID="de3d44f5eaa26f1b3f83fae9f500bc5e18ea75cb672b56e2d63581de848d89eb"
Jul 17 18:12:36.784036 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:36.784010    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"
Jul 17 18:12:36.925046 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:36.924963    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8a0993-6dd7-445f-8bc1-3b914db314b3-maas-api-service-ca\") pod \"dd8a0993-6dd7-445f-8bc1-3b914db314b3\" (UID: \"dd8a0993-6dd7-445f-8bc1-3b914db314b3\") "
Jul 17 18:12:36.925337 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:36.925313    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd8a0993-6dd7-445f-8bc1-3b914db314b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd8a0993-6dd7-445f-8bc1-3b914db314b3" (UID: "dd8a0993-6dd7-445f-8bc1-3b914db314b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:12:37.026692 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:37.026648    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8a0993-6dd7-445f-8bc1-3b914db314b3-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:37.648796 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:37.648764    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7"
Jul 17 18:12:37.649252 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:37.648763    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7" event={"ID":"dd8a0993-6dd7-445f-8bc1-3b914db314b3","Type":"ContainerDied","Data":"a2e37db4f2492ce9d75dc7c1ead9378955fb8afb04c5e751a13e84919ec0e7ff"}
Jul 17 18:12:37.649252 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:37.648904    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2e37db4f2492ce9d75dc7c1ead9378955fb8afb04c5e751a13e84919ec0e7ff"
Jul 17 18:12:37.650620 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:37.650597    2572 generic.go:356] "Generic (PLEG): container finished" podID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerID="8f5550627ee372fed3a72320ec2e5a7a4d034fb3636e4d6ce9db297afc3a9c2f" exitCode=6
Jul 17 18:12:37.650772 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:37.650635    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" event={"ID":"1182526d-6b1a-4aab-879f-5a1e9ab65da2","Type":"ContainerDied","Data":"8f5550627ee372fed3a72320ec2e5a7a4d034fb3636e4d6ce9db297afc3a9c2f"}
Jul 17 18:12:37.650772 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:37.650690    2572 scope.go:117] "RemoveContainer" containerID="de3d44f5eaa26f1b3f83fae9f500bc5e18ea75cb672b56e2d63581de848d89eb"
Jul 17 18:12:37.650918 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:37.650901    2572 scope.go:117] "RemoveContainer" containerID="8f5550627ee372fed3a72320ec2e5a7a4d034fb3636e4d6ce9db297afc3a9c2f"
Jul 17 18:12:37.651126 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:37.651106    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p_odh-ai-gateway-infra(1182526d-6b1a-4aab-879f-5a1e9ab65da2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2"
Jul 17 18:12:38.514267 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:38.514223    2572 scope.go:117] "RemoveContainer" containerID="062da7307cf64370950b21252ecf7eb2ce097d0ccb171af308ea683a1c70e1b9"
Jul 17 18:12:38.658264 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:38.658234    2572 scope.go:117] "RemoveContainer" containerID="8f5550627ee372fed3a72320ec2e5a7a4d034fb3636e4d6ce9db297afc3a9c2f"
Jul 17 18:12:38.658782 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:38.658456    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p_odh-ai-gateway-infra(1182526d-6b1a-4aab-879f-5a1e9ab65da2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2"
Jul 17 18:12:39.665445 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:39.665410    2572 generic.go:356] "Generic (PLEG): container finished" podID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerID="ff5bccef00de971c1350bd10a7e3dd1882c2c3203501015b7bbedde693784285" exitCode=6
Jul 17 18:12:39.665897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:39.665466    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv" event={"ID":"20333f6f-4c9e-4dc0-989e-52f471fa9559","Type":"ContainerDied","Data":"ff5bccef00de971c1350bd10a7e3dd1882c2c3203501015b7bbedde693784285"}
Jul 17 18:12:39.665897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:39.665546    2572 scope.go:117] "RemoveContainer" containerID="062da7307cf64370950b21252ecf7eb2ce097d0ccb171af308ea683a1c70e1b9"
Jul 17 18:12:39.665897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:39.665841    2572 scope.go:117] "RemoveContainer" containerID="ff5bccef00de971c1350bd10a7e3dd1882c2c3203501015b7bbedde693784285"
Jul 17 18:12:39.666077 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:39.666054    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv_odh-ai-gateway-infra(20333f6f-4c9e-4dc0-989e-52f471fa9559)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559"
Jul 17 18:12:40.702996 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:40.702959    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"]
Jul 17 18:12:40.845601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:40.845574    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"
Jul 17 18:12:40.963287 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:40.963186    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20333f6f-4c9e-4dc0-989e-52f471fa9559-maas-api-service-ca\") pod \"20333f6f-4c9e-4dc0-989e-52f471fa9559\" (UID: \"20333f6f-4c9e-4dc0-989e-52f471fa9559\") "
Jul 17 18:12:40.963616 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:40.963593    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20333f6f-4c9e-4dc0-989e-52f471fa9559-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "20333f6f-4c9e-4dc0-989e-52f471fa9559" (UID: "20333f6f-4c9e-4dc0-989e-52f471fa9559"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:12:41.064342 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:41.064304    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20333f6f-4c9e-4dc0-989e-52f471fa9559-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:41.683677 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:41.683641    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv" event={"ID":"20333f6f-4c9e-4dc0-989e-52f471fa9559","Type":"ContainerDied","Data":"8bfb26b917a4ddd2962ac9050b9b521f204da9172ae2ffb10a3791992213f09d"}
Jul 17 18:12:41.683677 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:41.683680    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"
Jul 17 18:12:41.683972 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:41.683706    2572 scope.go:117] "RemoveContainer" containerID="ff5bccef00de971c1350bd10a7e3dd1882c2c3203501015b7bbedde693784285"
Jul 17 18:12:41.708675 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:41.708594    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"]
Jul 17 18:12:41.710921 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:41.710894    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-mjmqv"]
Jul 17 18:12:42.515078 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:42.515044    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" path="/var/lib/kubelet/pods/20333f6f-4c9e-4dc0-989e-52f471fa9559/volumes"
Jul 17 18:12:46.776987 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.776948    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"]
Jul 17 18:12:46.777382 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777270    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd8a0993-6dd7-445f-8bc1-3b914db314b3" containerName="revoke-keys"
Jul 17 18:12:46.777382 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777281    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8a0993-6dd7-445f-8bc1-3b914db314b3" containerName="revoke-keys"
Jul 17 18:12:46.777382 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777297    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerName="revoke-keys"
Jul 17 18:12:46.777382 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777303    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerName="revoke-keys"
Jul 17 18:12:46.777382 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777319    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerName="revoke-keys"
Jul 17 18:12:46.777382 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777324    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerName="revoke-keys"
Jul 17 18:12:46.777382 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777332    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerName="revoke-keys"
Jul 17 18:12:46.777382 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777337    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerName="revoke-keys"
Jul 17 18:12:46.777668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777389    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerName="revoke-keys"
Jul 17 18:12:46.777668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777399    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerName="revoke-keys"
Jul 17 18:12:46.777668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777405    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd8a0993-6dd7-445f-8bc1-3b914db314b3" containerName="revoke-keys"
Jul 17 18:12:46.777668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.777411    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="20333f6f-4c9e-4dc0-989e-52f471fa9559" containerName="revoke-keys"
Jul 17 18:12:46.781788 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.781768    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"
Jul 17 18:12:46.789358 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.788841    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"]
Jul 17 18:12:46.917209 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:46.917168    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df82bda3-2846-4b5e-a9fa-ee8642f61ef9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-89bq7\" (UID: \"df82bda3-2846-4b5e-a9fa-ee8642f61ef9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"
Jul 17 18:12:47.018572 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:47.018529    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df82bda3-2846-4b5e-a9fa-ee8642f61ef9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-89bq7\" (UID: \"df82bda3-2846-4b5e-a9fa-ee8642f61ef9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"
Jul 17 18:12:47.019150 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:47.019130    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df82bda3-2846-4b5e-a9fa-ee8642f61ef9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-89bq7\" (UID: \"df82bda3-2846-4b5e-a9fa-ee8642f61ef9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"
Jul 17 18:12:47.092431 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:47.092396    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"
Jul 17 18:12:47.237444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:47.237420    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"]
Jul 17 18:12:47.239318 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:12:47.239286    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf82bda3_2846_4b5e_a9fa_ee8642f61ef9.slice/crio-d2f1ae6c280305383528f614fea9b227463e947b91d6fefac42b4a3047bfb41b WatchSource:0}: Error finding container d2f1ae6c280305383528f614fea9b227463e947b91d6fefac42b4a3047bfb41b: Status 404 returned error can't find the container with id d2f1ae6c280305383528f614fea9b227463e947b91d6fefac42b4a3047bfb41b
Jul 17 18:12:47.719478 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:47.719380    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" event={"ID":"df82bda3-2846-4b5e-a9fa-ee8642f61ef9","Type":"ContainerStarted","Data":"33e66fdc8c8ddef5a5daa9467089c539fda2b000712dbb9e1675e19074562ad6"}
Jul 17 18:12:47.719478 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:47.719435    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" event={"ID":"df82bda3-2846-4b5e-a9fa-ee8642f61ef9","Type":"ContainerStarted","Data":"d2f1ae6c280305383528f614fea9b227463e947b91d6fefac42b4a3047bfb41b"}
Jul 17 18:12:47.739237 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:47.739163    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" podStartSLOduration=1.7391397419999999 podStartE2EDuration="1.739139742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:12:47.73467283 +0000 UTC m=+2939.740028685" watchObservedRunningTime="2026-07-17 18:12:47.739139742 +0000 UTC m=+2939.744495597"
Jul 17 18:12:48.687878 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:48.687839    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"]
Jul 17 18:12:48.691533 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:48.691482    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"
Jul 17 18:12:48.700737 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:48.700687    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"]
Jul 17 18:12:48.727809 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:48.727770    2572 generic.go:356] "Generic (PLEG): container finished" podID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerID="33e66fdc8c8ddef5a5daa9467089c539fda2b000712dbb9e1675e19074562ad6" exitCode=6
Jul 17 18:12:48.728000 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:48.727849    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" event={"ID":"df82bda3-2846-4b5e-a9fa-ee8642f61ef9","Type":"ContainerDied","Data":"33e66fdc8c8ddef5a5daa9467089c539fda2b000712dbb9e1675e19074562ad6"}
Jul 17 18:12:48.728087 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:48.728063    2572 scope.go:117] "RemoveContainer" containerID="33e66fdc8c8ddef5a5daa9467089c539fda2b000712dbb9e1675e19074562ad6"
Jul 17 18:12:48.834721 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:48.834687    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85484e95-2314-47f1-961d-ad4313e5f309-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l\" (UID: \"85484e95-2314-47f1-961d-ad4313e5f309\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"
Jul 17 18:12:48.935612 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:48.935581    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85484e95-2314-47f1-961d-ad4313e5f309-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l\" (UID: \"85484e95-2314-47f1-961d-ad4313e5f309\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"
Jul 17 18:12:48.936178 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:48.936160    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85484e95-2314-47f1-961d-ad4313e5f309-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l\" (UID: \"85484e95-2314-47f1-961d-ad4313e5f309\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"
Jul 17 18:12:49.003837 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:49.003783    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"
Jul 17 18:12:49.154595 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:49.154566    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"]
Jul 17 18:12:49.157114 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:12:49.157077    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85484e95_2314_47f1_961d_ad4313e5f309.slice/crio-bd13afc0198737e0ff1303c3b858b10a985f8c25dd3f5d69f16bff749c09b5b2 WatchSource:0}: Error finding container bd13afc0198737e0ff1303c3b858b10a985f8c25dd3f5d69f16bff749c09b5b2: Status 404 returned error can't find the container with id bd13afc0198737e0ff1303c3b858b10a985f8c25dd3f5d69f16bff749c09b5b2
Jul 17 18:12:49.735412 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:49.735305    2572 generic.go:356] "Generic (PLEG): container finished" podID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerID="a93d994100d005b41d01894580896189f5f0cf653f2cd74099a8ef445b75e688" exitCode=6
Jul 17 18:12:49.735901 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:49.735435    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" event={"ID":"df82bda3-2846-4b5e-a9fa-ee8642f61ef9","Type":"ContainerDied","Data":"a93d994100d005b41d01894580896189f5f0cf653f2cd74099a8ef445b75e688"}
Jul 17 18:12:49.735901 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:49.735513    2572 scope.go:117] "RemoveContainer" containerID="33e66fdc8c8ddef5a5daa9467089c539fda2b000712dbb9e1675e19074562ad6"
Jul 17 18:12:49.735901 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:49.735725    2572 scope.go:117] "RemoveContainer" containerID="a93d994100d005b41d01894580896189f5f0cf653f2cd74099a8ef445b75e688"
Jul 17 18:12:49.736026 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:49.735934    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-89bq7_odh-ai-gateway-infra(df82bda3-2846-4b5e-a9fa-ee8642f61ef9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9"
Jul 17 18:12:49.737352 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:49.737326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" event={"ID":"85484e95-2314-47f1-961d-ad4313e5f309","Type":"ContainerStarted","Data":"49b7c287a4ffb8b4c0fdbfbe4c029a30a64de0f43cbaf4126b9147a8389c10d4"}
Jul 17 18:12:49.737466 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:49.737372    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" event={"ID":"85484e95-2314-47f1-961d-ad4313e5f309","Type":"ContainerStarted","Data":"bd13afc0198737e0ff1303c3b858b10a985f8c25dd3f5d69f16bff749c09b5b2"}
Jul 17 18:12:49.768206 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:49.768147    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" podStartSLOduration=1.768127437 podStartE2EDuration="1.768127437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:12:49.766834216 +0000 UTC m=+2941.772190069" watchObservedRunningTime="2026-07-17 18:12:49.768127437 +0000 UTC m=+2941.773483289"
Jul 17 18:12:50.744209 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:50.744173    2572 generic.go:356] "Generic (PLEG): container finished" podID="85484e95-2314-47f1-961d-ad4313e5f309" containerID="49b7c287a4ffb8b4c0fdbfbe4c029a30a64de0f43cbaf4126b9147a8389c10d4" exitCode=6
Jul 17 18:12:50.744693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:50.744273    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" event={"ID":"85484e95-2314-47f1-961d-ad4313e5f309","Type":"ContainerDied","Data":"49b7c287a4ffb8b4c0fdbfbe4c029a30a64de0f43cbaf4126b9147a8389c10d4"}
Jul 17 18:12:50.744693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:50.744440    2572 scope.go:117] "RemoveContainer" containerID="49b7c287a4ffb8b4c0fdbfbe4c029a30a64de0f43cbaf4126b9147a8389c10d4"
Jul 17 18:12:50.746219 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:50.746204    2572 scope.go:117] "RemoveContainer" containerID="a93d994100d005b41d01894580896189f5f0cf653f2cd74099a8ef445b75e688"
Jul 17 18:12:50.746389 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:50.746369    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-89bq7_odh-ai-gateway-infra(df82bda3-2846-4b5e-a9fa-ee8642f61ef9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9"
Jul 17 18:12:51.753356 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:51.753256    2572 generic.go:356] "Generic (PLEG): container finished" podID="85484e95-2314-47f1-961d-ad4313e5f309" containerID="05d5b385dfe026081bb342bca6b5148110e5da1300ca1ebff1890996f9619902" exitCode=6
Jul 17 18:12:51.753356 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:51.753319    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" event={"ID":"85484e95-2314-47f1-961d-ad4313e5f309","Type":"ContainerDied","Data":"05d5b385dfe026081bb342bca6b5148110e5da1300ca1ebff1890996f9619902"}
Jul 17 18:12:51.753846 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:51.753365    2572 scope.go:117] "RemoveContainer" containerID="49b7c287a4ffb8b4c0fdbfbe4c029a30a64de0f43cbaf4126b9147a8389c10d4"
Jul 17 18:12:51.753846 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:51.753562    2572 scope.go:117] "RemoveContainer" containerID="05d5b385dfe026081bb342bca6b5148110e5da1300ca1ebff1890996f9619902"
Jul 17 18:12:51.753846 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:51.753805    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l_odh-ai-gateway-infra(85484e95-2314-47f1-961d-ad4313e5f309)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" podUID="85484e95-2314-47f1-961d-ad4313e5f309"
Jul 17 18:12:52.510229 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:52.510191    2572 scope.go:117] "RemoveContainer" containerID="8f5550627ee372fed3a72320ec2e5a7a4d034fb3636e4d6ce9db297afc3a9c2f"
Jul 17 18:12:52.762907 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:52.762819    2572 scope.go:117] "RemoveContainer" containerID="05d5b385dfe026081bb342bca6b5148110e5da1300ca1ebff1890996f9619902"
Jul 17 18:12:52.763297 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:52.763038    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l_odh-ai-gateway-infra(85484e95-2314-47f1-961d-ad4313e5f309)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" podUID="85484e95-2314-47f1-961d-ad4313e5f309"
Jul 17 18:12:53.770801 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:53.770763    2572 generic.go:356] "Generic (PLEG): container finished" podID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerID="bc534faaa0a5d9776229f812e86f5c4fd3a70f52c8d6d57392a48044df70026f" exitCode=6
Jul 17 18:12:53.771235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:53.770831    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" event={"ID":"1182526d-6b1a-4aab-879f-5a1e9ab65da2","Type":"ContainerDied","Data":"bc534faaa0a5d9776229f812e86f5c4fd3a70f52c8d6d57392a48044df70026f"}
Jul 17 18:12:53.771235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:53.770921    2572 scope.go:117] "RemoveContainer" containerID="8f5550627ee372fed3a72320ec2e5a7a4d034fb3636e4d6ce9db297afc3a9c2f"
Jul 17 18:12:53.771332 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:53.771277    2572 scope.go:117] "RemoveContainer" containerID="bc534faaa0a5d9776229f812e86f5c4fd3a70f52c8d6d57392a48044df70026f"
Jul 17 18:12:53.771574 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:12:53.771552    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p_odh-ai-gateway-infra(1182526d-6b1a-4aab-879f-5a1e9ab65da2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2"
Jul 17 18:12:54.800312 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:54.800273    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"]
Jul 17 18:12:54.950911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:54.950838    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"
Jul 17 18:12:54.988586 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:54.988554    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1182526d-6b1a-4aab-879f-5a1e9ab65da2-maas-api-service-ca\") pod \"1182526d-6b1a-4aab-879f-5a1e9ab65da2\" (UID: \"1182526d-6b1a-4aab-879f-5a1e9ab65da2\") "
Jul 17 18:12:54.988931 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:54.988910    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1182526d-6b1a-4aab-879f-5a1e9ab65da2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1182526d-6b1a-4aab-879f-5a1e9ab65da2" (UID: "1182526d-6b1a-4aab-879f-5a1e9ab65da2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:12:55.089166 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:55.089127    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1182526d-6b1a-4aab-879f-5a1e9ab65da2-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:12:55.785677 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:55.785637    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p" event={"ID":"1182526d-6b1a-4aab-879f-5a1e9ab65da2","Type":"ContainerDied","Data":"a7679f09412d93d5dc2d5fe23591d59bf355b1cafbe1850b10c268f7128f75d5"}
Jul 17 18:12:55.785893 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:55.785707    2572 scope.go:117] "RemoveContainer" containerID="bc534faaa0a5d9776229f812e86f5c4fd3a70f52c8d6d57392a48044df70026f"
Jul 17 18:12:55.785893 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:55.785650    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"
Jul 17 18:12:55.808951 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:55.808916    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"]
Jul 17 18:12:55.816589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:55.816559    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-rxr9p"]
Jul 17 18:12:56.516440 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:12:56.516395    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" path="/var/lib/kubelet/pods/1182526d-6b1a-4aab-879f-5a1e9ab65da2/volumes"
Jul 17 18:13:04.510647 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:04.510613    2572 scope.go:117] "RemoveContainer" containerID="a93d994100d005b41d01894580896189f5f0cf653f2cd74099a8ef445b75e688"
Jul 17 18:13:05.848398 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:05.848363    2572 generic.go:356] "Generic (PLEG): container finished" podID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerID="c0a490e2f7672b83a5d8dd49c44e4c8b8f254e0d4f703a80b9d4717d6b7e433e" exitCode=6
Jul 17 18:13:05.848911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:05.848422    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" event={"ID":"df82bda3-2846-4b5e-a9fa-ee8642f61ef9","Type":"ContainerDied","Data":"c0a490e2f7672b83a5d8dd49c44e4c8b8f254e0d4f703a80b9d4717d6b7e433e"}
Jul 17 18:13:05.848911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:05.848482    2572 scope.go:117] "RemoveContainer" containerID="a93d994100d005b41d01894580896189f5f0cf653f2cd74099a8ef445b75e688"
Jul 17 18:13:05.848911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:05.848851    2572 scope.go:117] "RemoveContainer" containerID="c0a490e2f7672b83a5d8dd49c44e4c8b8f254e0d4f703a80b9d4717d6b7e433e"
Jul 17 18:13:05.849137 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:05.849116    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-89bq7_odh-ai-gateway-infra(df82bda3-2846-4b5e-a9fa-ee8642f61ef9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9"
Jul 17 18:13:06.510585 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:06.510546    2572 scope.go:117] "RemoveContainer" containerID="05d5b385dfe026081bb342bca6b5148110e5da1300ca1ebff1890996f9619902"
Jul 17 18:13:06.857394 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:06.857352    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" event={"ID":"85484e95-2314-47f1-961d-ad4313e5f309","Type":"ContainerStarted","Data":"9a571afd000b03ad021dc21ee290b1accf2bdca5eb98d401866ebc95057ed9a9"}
Jul 17 18:13:06.877316 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:06.877250    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"]
Jul 17 18:13:07.016147 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.016120    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"
Jul 17 18:13:07.085294 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.085257    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df82bda3-2846-4b5e-a9fa-ee8642f61ef9-maas-api-service-ca\") pod \"df82bda3-2846-4b5e-a9fa-ee8642f61ef9\" (UID: \"df82bda3-2846-4b5e-a9fa-ee8642f61ef9\") "
Jul 17 18:13:07.085719 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.085695    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df82bda3-2846-4b5e-a9fa-ee8642f61ef9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df82bda3-2846-4b5e-a9fa-ee8642f61ef9" (UID: "df82bda3-2846-4b5e-a9fa-ee8642f61ef9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:13:07.186772 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.186728    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df82bda3-2846-4b5e-a9fa-ee8642f61ef9-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:13:07.864452 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.864416    2572 generic.go:356] "Generic (PLEG): container finished" podID="85484e95-2314-47f1-961d-ad4313e5f309" containerID="9a571afd000b03ad021dc21ee290b1accf2bdca5eb98d401866ebc95057ed9a9" exitCode=6
Jul 17 18:13:07.864945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.864508    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" event={"ID":"85484e95-2314-47f1-961d-ad4313e5f309","Type":"ContainerDied","Data":"9a571afd000b03ad021dc21ee290b1accf2bdca5eb98d401866ebc95057ed9a9"}
Jul 17 18:13:07.864945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.864605    2572 scope.go:117] "RemoveContainer" containerID="05d5b385dfe026081bb342bca6b5148110e5da1300ca1ebff1890996f9619902"
Jul 17 18:13:07.864945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.864928    2572 scope.go:117] "RemoveContainer" containerID="9a571afd000b03ad021dc21ee290b1accf2bdca5eb98d401866ebc95057ed9a9"
Jul 17 18:13:07.865177 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:07.865153    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l_odh-ai-gateway-infra(85484e95-2314-47f1-961d-ad4313e5f309)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" podUID="85484e95-2314-47f1-961d-ad4313e5f309"
Jul 17 18:13:07.865896 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.865873    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"
Jul 17 18:13:07.865998 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.865866    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7" event={"ID":"df82bda3-2846-4b5e-a9fa-ee8642f61ef9","Type":"ContainerDied","Data":"d2f1ae6c280305383528f614fea9b227463e947b91d6fefac42b4a3047bfb41b"}
Jul 17 18:13:07.876713 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.876693    2572 scope.go:117] "RemoveContainer" containerID="c0a490e2f7672b83a5d8dd49c44e4c8b8f254e0d4f703a80b9d4717d6b7e433e"
Jul 17 18:13:07.895987 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.895955    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"]
Jul 17 18:13:07.907922 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.907879    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"]
Jul 17 18:13:07.911372 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:07.911341    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-89bq7"]
Jul 17 18:13:08.516002 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:08.515966    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" path="/var/lib/kubelet/pods/df82bda3-2846-4b5e-a9fa-ee8642f61ef9/volumes"
Jul 17 18:13:09.015986 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:09.015958    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"
Jul 17 18:13:09.102516 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:09.102458    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85484e95-2314-47f1-961d-ad4313e5f309-maas-api-service-ca\") pod \"85484e95-2314-47f1-961d-ad4313e5f309\" (UID: \"85484e95-2314-47f1-961d-ad4313e5f309\") "
Jul 17 18:13:09.102924 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:09.102897    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85484e95-2314-47f1-961d-ad4313e5f309-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85484e95-2314-47f1-961d-ad4313e5f309" (UID: "85484e95-2314-47f1-961d-ad4313e5f309"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:13:09.203634 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:09.203526    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85484e95-2314-47f1-961d-ad4313e5f309-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:13:09.881038 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:09.881009    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"
Jul 17 18:13:09.881341 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:09.881046    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l" event={"ID":"85484e95-2314-47f1-961d-ad4313e5f309","Type":"ContainerDied","Data":"bd13afc0198737e0ff1303c3b858b10a985f8c25dd3f5d69f16bff749c09b5b2"}
Jul 17 18:13:09.881341 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:09.881110    2572 scope.go:117] "RemoveContainer" containerID="9a571afd000b03ad021dc21ee290b1accf2bdca5eb98d401866ebc95057ed9a9"
Jul 17 18:13:09.907193 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:09.907154    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"]
Jul 17 18:13:09.912891 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:09.912853    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-nzc2l"]
Jul 17 18:13:10.516616 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:10.516541    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85484e95-2314-47f1-961d-ad4313e5f309" path="/var/lib/kubelet/pods/85484e95-2314-47f1-961d-ad4313e5f309/volumes"
Jul 17 18:13:20.077795 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.077758    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"]
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078091    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85484e95-2314-47f1-961d-ad4313e5f309" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078102    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="85484e95-2314-47f1-961d-ad4313e5f309" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078109    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078118    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078130    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078135    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078143    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078148    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078160    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078165    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078173    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85484e95-2314-47f1-961d-ad4313e5f309" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078178    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="85484e95-2314-47f1-961d-ad4313e5f309" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078230    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="85484e95-2314-47f1-961d-ad4313e5f309" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078236    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerName="revoke-keys"
Jul 17 18:13:20.078235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078243    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerName="revoke-keys"
Jul 17 18:13:20.078731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078249    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerName="revoke-keys"
Jul 17 18:13:20.078731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078253    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="85484e95-2314-47f1-961d-ad4313e5f309" containerName="revoke-keys"
Jul 17 18:13:20.078731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078259    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerName="revoke-keys"
Jul 17 18:13:20.078731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078264    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="85484e95-2314-47f1-961d-ad4313e5f309" containerName="revoke-keys"
Jul 17 18:13:20.078731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.078270    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerName="revoke-keys"
Jul 17 18:13:20.082631 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.082605    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"
Jul 17 18:13:20.091135 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.091105    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:13:20.092283 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.092259    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:13:20.094882 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.094851    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"]
Jul 17 18:13:20.200968 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.200935    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11028558-6d0e-4870-9a04-bf32911ac0ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6\" (UID: \"11028558-6d0e-4870-9a04-bf32911ac0ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"
Jul 17 18:13:20.301909 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.301860    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11028558-6d0e-4870-9a04-bf32911ac0ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6\" (UID: \"11028558-6d0e-4870-9a04-bf32911ac0ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"
Jul 17 18:13:20.302515 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.302467    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11028558-6d0e-4870-9a04-bf32911ac0ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6\" (UID: \"11028558-6d0e-4870-9a04-bf32911ac0ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"
Jul 17 18:13:20.402204 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.402103    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"
Jul 17 18:13:20.541741 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.541713    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"]
Jul 17 18:13:20.947750 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.947713    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" event={"ID":"11028558-6d0e-4870-9a04-bf32911ac0ec","Type":"ContainerStarted","Data":"0fefef0ece649b4a254ed521ed96d69ef42941d8a483b130f205c2def0b597da"}
Jul 17 18:13:20.947953 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.947773    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" event={"ID":"11028558-6d0e-4870-9a04-bf32911ac0ec","Type":"ContainerStarted","Data":"54166b2486f27306b6820f9b39879df458d55cfc8776a1d3901fc52bbeeedaae"}
Jul 17 18:13:20.965461 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:20.965403    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" podStartSLOduration=0.965384408 podStartE2EDuration="965.384408ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:13:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:13:20.963332386 +0000 UTC m=+2972.968688260" watchObservedRunningTime="2026-07-17 18:13:20.965384408 +0000 UTC m=+2972.970740260"
Jul 17 18:13:21.954047 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:21.954016    2572 generic.go:356] "Generic (PLEG): container finished" podID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerID="0fefef0ece649b4a254ed521ed96d69ef42941d8a483b130f205c2def0b597da" exitCode=6
Jul 17 18:13:21.954517 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:21.954104    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" event={"ID":"11028558-6d0e-4870-9a04-bf32911ac0ec","Type":"ContainerDied","Data":"0fefef0ece649b4a254ed521ed96d69ef42941d8a483b130f205c2def0b597da"}
Jul 17 18:13:21.954517 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:21.954276    2572 scope.go:117] "RemoveContainer" containerID="0fefef0ece649b4a254ed521ed96d69ef42941d8a483b130f205c2def0b597da"
Jul 17 18:13:22.962531 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:22.962393    2572 generic.go:356] "Generic (PLEG): container finished" podID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerID="8ea3ef6d8507c99dd6f6127d8438366da3b8a1539cde9b1b9fa2c7185e7dc97c" exitCode=6
Jul 17 18:13:22.962531 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:22.962474    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" event={"ID":"11028558-6d0e-4870-9a04-bf32911ac0ec","Type":"ContainerDied","Data":"8ea3ef6d8507c99dd6f6127d8438366da3b8a1539cde9b1b9fa2c7185e7dc97c"}
Jul 17 18:13:22.962978 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:22.962558    2572 scope.go:117] "RemoveContainer" containerID="0fefef0ece649b4a254ed521ed96d69ef42941d8a483b130f205c2def0b597da"
Jul 17 18:13:22.962978 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:22.962744    2572 scope.go:117] "RemoveContainer" containerID="8ea3ef6d8507c99dd6f6127d8438366da3b8a1539cde9b1b9fa2c7185e7dc97c"
Jul 17 18:13:22.963059 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:22.962997    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6_odh-ai-gateway-infra(11028558-6d0e-4870-9a04-bf32911ac0ec)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec"
Jul 17 18:13:23.970238 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:23.970210    2572 scope.go:117] "RemoveContainer" containerID="8ea3ef6d8507c99dd6f6127d8438366da3b8a1539cde9b1b9fa2c7185e7dc97c"
Jul 17 18:13:23.970664 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:23.970417    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6_odh-ai-gateway-infra(11028558-6d0e-4870-9a04-bf32911ac0ec)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec"
Jul 17 18:13:24.212441 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.212405    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"]
Jul 17 18:13:24.212773 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.212760    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85484e95-2314-47f1-961d-ad4313e5f309" containerName="revoke-keys"
Jul 17 18:13:24.212821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.212775    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="85484e95-2314-47f1-961d-ad4313e5f309" containerName="revoke-keys"
Jul 17 18:13:24.212821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.212783    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerName="revoke-keys"
Jul 17 18:13:24.212821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.212788    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerName="revoke-keys"
Jul 17 18:13:24.212821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.212803    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerName="revoke-keys"
Jul 17 18:13:24.212821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.212809    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1182526d-6b1a-4aab-879f-5a1e9ab65da2" containerName="revoke-keys"
Jul 17 18:13:24.212974 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.212881    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="df82bda3-2846-4b5e-a9fa-ee8642f61ef9" containerName="revoke-keys"
Jul 17 18:13:24.217155 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.217136    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"
Jul 17 18:13:24.225085 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.225056    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"]
Jul 17 18:13:24.340077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.340032    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60117f4f-e445-4065-9c9a-ba5a9b497653-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a27826-2md7x\" (UID: \"60117f4f-e445-4065-9c9a-ba5a9b497653\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"
Jul 17 18:13:24.441014 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.440974    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60117f4f-e445-4065-9c9a-ba5a9b497653-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a27826-2md7x\" (UID: \"60117f4f-e445-4065-9c9a-ba5a9b497653\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"
Jul 17 18:13:24.441628 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.441608    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60117f4f-e445-4065-9c9a-ba5a9b497653-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a27826-2md7x\" (UID: \"60117f4f-e445-4065-9c9a-ba5a9b497653\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"
Jul 17 18:13:24.530115 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.530012    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"
Jul 17 18:13:24.677545 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.676863    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"]
Jul 17 18:13:24.680203 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:13:24.680170    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60117f4f_e445_4065_9c9a_ba5a9b497653.slice/crio-a7af531346748052648600b40a86b0febdec8428b9e0ec553f60d9cd3e864ae4 WatchSource:0}: Error finding container a7af531346748052648600b40a86b0febdec8428b9e0ec553f60d9cd3e864ae4: Status 404 returned error can't find the container with id a7af531346748052648600b40a86b0febdec8428b9e0ec553f60d9cd3e864ae4
Jul 17 18:13:24.977510 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.977460    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x" event={"ID":"60117f4f-e445-4065-9c9a-ba5a9b497653","Type":"ContainerStarted","Data":"a9003f6cb7d8970d27c6f79bfd42ab65a096c75479aafdd2608667e44782bdcd"}
Jul 17 18:13:24.977916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.977541    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x" event={"ID":"60117f4f-e445-4065-9c9a-ba5a9b497653","Type":"ContainerStarted","Data":"a7af531346748052648600b40a86b0febdec8428b9e0ec553f60d9cd3e864ae4"}
Jul 17 18:13:24.994317 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:24.994041    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x" podStartSLOduration=0.994018023 podStartE2EDuration="994.018023ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:13:24.993400003 +0000 UTC m=+2976.998755859" watchObservedRunningTime="2026-07-17 18:13:24.994018023 +0000 UTC m=+2976.999373917"
Jul 17 18:13:25.984928 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:25.984892    2572 generic.go:356] "Generic (PLEG): container finished" podID="60117f4f-e445-4065-9c9a-ba5a9b497653" containerID="a9003f6cb7d8970d27c6f79bfd42ab65a096c75479aafdd2608667e44782bdcd" exitCode=0
Jul 17 18:13:25.985342 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:25.984987    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x" event={"ID":"60117f4f-e445-4065-9c9a-ba5a9b497653","Type":"ContainerDied","Data":"a9003f6cb7d8970d27c6f79bfd42ab65a096c75479aafdd2608667e44782bdcd"}
Jul 17 18:13:27.128867 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:27.128835    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"
Jul 17 18:13:27.267683 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:27.267583    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60117f4f-e445-4065-9c9a-ba5a9b497653-maas-api-service-ca\") pod \"60117f4f-e445-4065-9c9a-ba5a9b497653\" (UID: \"60117f4f-e445-4065-9c9a-ba5a9b497653\") "
Jul 17 18:13:27.267973 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:27.267949    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60117f4f-e445-4065-9c9a-ba5a9b497653-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "60117f4f-e445-4065-9c9a-ba5a9b497653" (UID: "60117f4f-e445-4065-9c9a-ba5a9b497653"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:13:27.369081 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:27.369038    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60117f4f-e445-4065-9c9a-ba5a9b497653-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:13:27.997557 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:27.997523    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x"
Jul 17 18:13:27.997755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:27.997529    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a27826-2md7x" event={"ID":"60117f4f-e445-4065-9c9a-ba5a9b497653","Type":"ContainerDied","Data":"a7af531346748052648600b40a86b0febdec8428b9e0ec553f60d9cd3e864ae4"}
Jul 17 18:13:27.997755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:27.997655    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7af531346748052648600b40a86b0febdec8428b9e0ec553f60d9cd3e864ae4"
Jul 17 18:13:34.596061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.596022    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"]
Jul 17 18:13:34.596529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.596379    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60117f4f-e445-4065-9c9a-ba5a9b497653" containerName="revoke-keys"
Jul 17 18:13:34.596529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.596390    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="60117f4f-e445-4065-9c9a-ba5a9b497653" containerName="revoke-keys"
Jul 17 18:13:34.596529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.596455    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="60117f4f-e445-4065-9c9a-ba5a9b497653" containerName="revoke-keys"
Jul 17 18:13:34.599553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.599533    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"
Jul 17 18:13:34.619123 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.619083    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"]
Jul 17 18:13:34.738901 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.738861    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5cabdb-7cc4-4122-8233-15ff6d13fd32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5\" (UID: \"1c5cabdb-7cc4-4122-8233-15ff6d13fd32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"
Jul 17 18:13:34.839457 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.839413    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5cabdb-7cc4-4122-8233-15ff6d13fd32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5\" (UID: \"1c5cabdb-7cc4-4122-8233-15ff6d13fd32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"
Jul 17 18:13:34.840096 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.840066    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5cabdb-7cc4-4122-8233-15ff6d13fd32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5\" (UID: \"1c5cabdb-7cc4-4122-8233-15ff6d13fd32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"
Jul 17 18:13:34.909865 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:34.909761    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"
Jul 17 18:13:35.053580 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:35.053546    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"]
Jul 17 18:13:35.055186 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:13:35.055161    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c5cabdb_7cc4_4122_8233_15ff6d13fd32.slice/crio-9c33f45dc214f0b1bf151db1d4e5c3265d654a0eb4bb5bb8a802aa7b9a7cf4d8 WatchSource:0}: Error finding container 9c33f45dc214f0b1bf151db1d4e5c3265d654a0eb4bb5bb8a802aa7b9a7cf4d8: Status 404 returned error can't find the container with id 9c33f45dc214f0b1bf151db1d4e5c3265d654a0eb4bb5bb8a802aa7b9a7cf4d8
Jul 17 18:13:36.044318 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:36.044225    2572 generic.go:356] "Generic (PLEG): container finished" podID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerID="78078b259f187991af40d4a09626a02725c60ed0500200dabdce73f481f02dd0" exitCode=6
Jul 17 18:13:36.044734 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:36.044311    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5" event={"ID":"1c5cabdb-7cc4-4122-8233-15ff6d13fd32","Type":"ContainerDied","Data":"78078b259f187991af40d4a09626a02725c60ed0500200dabdce73f481f02dd0"}
Jul 17 18:13:36.044734 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:36.044369    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5" event={"ID":"1c5cabdb-7cc4-4122-8233-15ff6d13fd32","Type":"ContainerStarted","Data":"9c33f45dc214f0b1bf151db1d4e5c3265d654a0eb4bb5bb8a802aa7b9a7cf4d8"}
Jul 17 18:13:36.044734 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:36.044517    2572 scope.go:117] "RemoveContainer" containerID="78078b259f187991af40d4a09626a02725c60ed0500200dabdce73f481f02dd0"
Jul 17 18:13:37.052113 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:37.052078    2572 generic.go:356] "Generic (PLEG): container finished" podID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerID="1278db6d0e3d3ca0431b82b0041fcf2eae7ea04dd095ec9e11380285f037302e" exitCode=6
Jul 17 18:13:37.052589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:37.052131    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5" event={"ID":"1c5cabdb-7cc4-4122-8233-15ff6d13fd32","Type":"ContainerDied","Data":"1278db6d0e3d3ca0431b82b0041fcf2eae7ea04dd095ec9e11380285f037302e"}
Jul 17 18:13:37.053061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:37.052206    2572 scope.go:117] "RemoveContainer" containerID="78078b259f187991af40d4a09626a02725c60ed0500200dabdce73f481f02dd0"
Jul 17 18:13:37.053139 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:37.053129    2572 scope.go:117] "RemoveContainer" containerID="1278db6d0e3d3ca0431b82b0041fcf2eae7ea04dd095ec9e11380285f037302e"
Jul 17 18:13:37.053431 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:37.053406    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5_odh-ai-gateway-infra(1c5cabdb-7cc4-4122-8233-15ff6d13fd32)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32"
Jul 17 18:13:37.509894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:37.509857    2572 scope.go:117] "RemoveContainer" containerID="8ea3ef6d8507c99dd6f6127d8438366da3b8a1539cde9b1b9fa2c7185e7dc97c"
Jul 17 18:13:38.060027 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:38.059999    2572 scope.go:117] "RemoveContainer" containerID="1278db6d0e3d3ca0431b82b0041fcf2eae7ea04dd095ec9e11380285f037302e"
Jul 17 18:13:38.060526 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:38.060233    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5_odh-ai-gateway-infra(1c5cabdb-7cc4-4122-8233-15ff6d13fd32)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32"
Jul 17 18:13:38.061977 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:38.061941    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" event={"ID":"11028558-6d0e-4870-9a04-bf32911ac0ec","Type":"ContainerStarted","Data":"b91287f8045eef4374437820a94fea86f9cec46566149582b3da6973201722b9"}
Jul 17 18:13:39.069875 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:39.069837    2572 generic.go:356] "Generic (PLEG): container finished" podID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerID="b91287f8045eef4374437820a94fea86f9cec46566149582b3da6973201722b9" exitCode=6
Jul 17 18:13:39.070315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:39.069881    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" event={"ID":"11028558-6d0e-4870-9a04-bf32911ac0ec","Type":"ContainerDied","Data":"b91287f8045eef4374437820a94fea86f9cec46566149582b3da6973201722b9"}
Jul 17 18:13:39.070315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:39.069934    2572 scope.go:117] "RemoveContainer" containerID="8ea3ef6d8507c99dd6f6127d8438366da3b8a1539cde9b1b9fa2c7185e7dc97c"
Jul 17 18:13:39.070315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:39.070280    2572 scope.go:117] "RemoveContainer" containerID="b91287f8045eef4374437820a94fea86f9cec46566149582b3da6973201722b9"
Jul 17 18:13:39.070563 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:39.070537    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6_odh-ai-gateway-infra(11028558-6d0e-4870-9a04-bf32911ac0ec)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec"
Jul 17 18:13:39.102542 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:39.102503    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"]
Jul 17 18:13:40.221587 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:40.221548    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"
Jul 17 18:13:40.394786 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:40.394674    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11028558-6d0e-4870-9a04-bf32911ac0ec-maas-api-service-ca\") pod \"11028558-6d0e-4870-9a04-bf32911ac0ec\" (UID: \"11028558-6d0e-4870-9a04-bf32911ac0ec\") "
Jul 17 18:13:40.395041 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:40.395018    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11028558-6d0e-4870-9a04-bf32911ac0ec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11028558-6d0e-4870-9a04-bf32911ac0ec" (UID: "11028558-6d0e-4870-9a04-bf32911ac0ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:13:40.496078 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:40.496041    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11028558-6d0e-4870-9a04-bf32911ac0ec-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:13:41.084417 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:41.084388    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"
Jul 17 18:13:41.084746 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:41.084383    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6" event={"ID":"11028558-6d0e-4870-9a04-bf32911ac0ec","Type":"ContainerDied","Data":"54166b2486f27306b6820f9b39879df458d55cfc8776a1d3901fc52bbeeedaae"}
Jul 17 18:13:41.084746 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:41.084546    2572 scope.go:117] "RemoveContainer" containerID="b91287f8045eef4374437820a94fea86f9cec46566149582b3da6973201722b9"
Jul 17 18:13:41.104254 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:41.104222    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"]
Jul 17 18:13:41.114164 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:41.114130    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-cq2z6"]
Jul 17 18:13:42.515679 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:42.515646    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" path="/var/lib/kubelet/pods/11028558-6d0e-4870-9a04-bf32911ac0ec/volumes"
Jul 17 18:13:46.827105 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.827059    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"]
Jul 17 18:13:46.827630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.827575    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerName="revoke-keys"
Jul 17 18:13:46.827630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.827598    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerName="revoke-keys"
Jul 17 18:13:46.827630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.827624    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerName="revoke-keys"
Jul 17 18:13:46.827796 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.827632    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerName="revoke-keys"
Jul 17 18:13:46.827796 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.827745    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerName="revoke-keys"
Jul 17 18:13:46.827796 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.827758    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerName="revoke-keys"
Jul 17 18:13:46.832173 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.832146    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"
Jul 17 18:13:46.841575 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.841541    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"]
Jul 17 18:13:46.851321 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.851287    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d4315b-4fc4-4760-ac16-5706941e2b39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g\" (UID: \"f7d4315b-4fc4-4760-ac16-5706941e2b39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"
Jul 17 18:13:46.952265 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.952223    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d4315b-4fc4-4760-ac16-5706941e2b39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g\" (UID: \"f7d4315b-4fc4-4760-ac16-5706941e2b39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"
Jul 17 18:13:46.952884 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:46.952862    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d4315b-4fc4-4760-ac16-5706941e2b39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g\" (UID: \"f7d4315b-4fc4-4760-ac16-5706941e2b39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"
Jul 17 18:13:47.143781 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:47.143690    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"
Jul 17 18:13:47.281213 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:47.281176    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"]
Jul 17 18:13:47.282588 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:13:47.282560    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d4315b_4fc4_4760_ac16_5706941e2b39.slice/crio-6df6db4e5c0950b3b3f7f6a513ee15db6bcff09bd3e897e0e7c09b41f7817de2 WatchSource:0}: Error finding container 6df6db4e5c0950b3b3f7f6a513ee15db6bcff09bd3e897e0e7c09b41f7817de2: Status 404 returned error can't find the container with id 6df6db4e5c0950b3b3f7f6a513ee15db6bcff09bd3e897e0e7c09b41f7817de2
Jul 17 18:13:48.132909 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.132867    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" event={"ID":"f7d4315b-4fc4-4760-ac16-5706941e2b39","Type":"ContainerStarted","Data":"7214c83f135223adc6dfb68fd4b178a012e42554af6249893731cfaa141dc3e3"}
Jul 17 18:13:48.133414 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.132981    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" event={"ID":"f7d4315b-4fc4-4760-ac16-5706941e2b39","Type":"ContainerStarted","Data":"6df6db4e5c0950b3b3f7f6a513ee15db6bcff09bd3e897e0e7c09b41f7817de2"}
Jul 17 18:13:48.157523 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.157450    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" podStartSLOduration=2.157433774 podStartE2EDuration="2.157433774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:13:48.155084002 +0000 UTC m=+3000.160439856" watchObservedRunningTime="2026-07-17 18:13:48.157433774 +0000 UTC m=+3000.162789689"
Jul 17 18:13:48.730237 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.730203    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 18:13:48.734278 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.734238    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"]
Jul 17 18:13:48.734760 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.734726    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerName="revoke-keys"
Jul 17 18:13:48.734760 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.734748    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerName="revoke-keys"
Jul 17 18:13:48.734989 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.734827    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="11028558-6d0e-4870-9a04-bf32911ac0ec" containerName="revoke-keys"
Jul 17 18:13:48.738009 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.737984    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 18:13:48.738156 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.738100    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"
Jul 17 18:13:48.738451 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.738415    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 18:13:48.745601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.745556    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"]
Jul 17 18:13:48.749005 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.748982    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 18:13:48.767069 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.767030    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b964fcbf-51a7-4ac1-9839-9fd11facdc90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9\" (UID: \"b964fcbf-51a7-4ac1-9839-9fd11facdc90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"
Jul 17 18:13:48.868161 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.868114    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b964fcbf-51a7-4ac1-9839-9fd11facdc90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9\" (UID: \"b964fcbf-51a7-4ac1-9839-9fd11facdc90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"
Jul 17 18:13:48.868770 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:48.868752    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b964fcbf-51a7-4ac1-9839-9fd11facdc90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9\" (UID: \"b964fcbf-51a7-4ac1-9839-9fd11facdc90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"
Jul 17 18:13:49.053264 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:49.053158    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"
Jul 17 18:13:49.140435 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:49.140401    2572 generic.go:356] "Generic (PLEG): container finished" podID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerID="7214c83f135223adc6dfb68fd4b178a012e42554af6249893731cfaa141dc3e3" exitCode=6
Jul 17 18:13:49.140891 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:49.140479    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" event={"ID":"f7d4315b-4fc4-4760-ac16-5706941e2b39","Type":"ContainerDied","Data":"7214c83f135223adc6dfb68fd4b178a012e42554af6249893731cfaa141dc3e3"}
Jul 17 18:13:49.140891 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:49.140756    2572 scope.go:117] "RemoveContainer" containerID="7214c83f135223adc6dfb68fd4b178a012e42554af6249893731cfaa141dc3e3"
Jul 17 18:13:49.197418 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:49.197389    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"]
Jul 17 18:13:49.198952 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:13:49.198916    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb964fcbf_51a7_4ac1_9839_9fd11facdc90.slice/crio-80d95701f94c6e795910c17a41dbd973fb1db8738c1d57a20baf49452cf93e76 WatchSource:0}: Error finding container 80d95701f94c6e795910c17a41dbd973fb1db8738c1d57a20baf49452cf93e76: Status 404 returned error can't find the container with id 80d95701f94c6e795910c17a41dbd973fb1db8738c1d57a20baf49452cf93e76
Jul 17 18:13:50.147775 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.147736    2572 generic.go:356] "Generic (PLEG): container finished" podID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerID="5c863443c8959b08021582a914afe4f82bde70d9f86878402bc8b4d7e81b90c7" exitCode=6
Jul 17 18:13:50.148274 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.147835    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9" event={"ID":"b964fcbf-51a7-4ac1-9839-9fd11facdc90","Type":"ContainerDied","Data":"5c863443c8959b08021582a914afe4f82bde70d9f86878402bc8b4d7e81b90c7"}
Jul 17 18:13:50.148274 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.147946    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9" event={"ID":"b964fcbf-51a7-4ac1-9839-9fd11facdc90","Type":"ContainerStarted","Data":"80d95701f94c6e795910c17a41dbd973fb1db8738c1d57a20baf49452cf93e76"}
Jul 17 18:13:50.148274 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.148067    2572 scope.go:117] "RemoveContainer" containerID="5c863443c8959b08021582a914afe4f82bde70d9f86878402bc8b4d7e81b90c7"
Jul 17 18:13:50.153556 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.153424    2572 generic.go:356] "Generic (PLEG): container finished" podID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerID="e388d8e003bfb00356757aa8dca4b12c5eb7c5ee18e3d09a9cc20cb1386fad59" exitCode=6
Jul 17 18:13:50.153680 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.153523    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" event={"ID":"f7d4315b-4fc4-4760-ac16-5706941e2b39","Type":"ContainerDied","Data":"e388d8e003bfb00356757aa8dca4b12c5eb7c5ee18e3d09a9cc20cb1386fad59"}
Jul 17 18:13:50.153758 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.153742    2572 scope.go:117] "RemoveContainer" containerID="7214c83f135223adc6dfb68fd4b178a012e42554af6249893731cfaa141dc3e3"
Jul 17 18:13:50.153862 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.153843    2572 scope.go:117] "RemoveContainer" containerID="e388d8e003bfb00356757aa8dca4b12c5eb7c5ee18e3d09a9cc20cb1386fad59"
Jul 17 18:13:50.154094 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:50.154070    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g_odh-ai-gateway-infra(f7d4315b-4fc4-4760-ac16-5706941e2b39)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39"
Jul 17 18:13:50.366897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.366856    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"]
Jul 17 18:13:50.371657 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.371634    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"
Jul 17 18:13:50.379451 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.379417    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"]
Jul 17 18:13:50.380535 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.380483    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d419b7-2667-477c-85c0-de7f88bb23ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b\" (UID: \"09d419b7-2667-477c-85c0-de7f88bb23ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"
Jul 17 18:13:50.481482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.481366    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d419b7-2667-477c-85c0-de7f88bb23ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b\" (UID: \"09d419b7-2667-477c-85c0-de7f88bb23ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"
Jul 17 18:13:50.482090 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.482062    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d419b7-2667-477c-85c0-de7f88bb23ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b\" (UID: \"09d419b7-2667-477c-85c0-de7f88bb23ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"
Jul 17 18:13:50.685089 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.685040    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"
Jul 17 18:13:50.828948 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:50.828916    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"]
Jul 17 18:13:50.829736 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:13:50.829704    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d419b7_2667_477c_85c0_de7f88bb23ee.slice/crio-b0dcd0a8bbb8507b3dbf2a2e0a2574ca022cd899f721c58dfa8a531ae1d4564f WatchSource:0}: Error finding container b0dcd0a8bbb8507b3dbf2a2e0a2574ca022cd899f721c58dfa8a531ae1d4564f: Status 404 returned error can't find the container with id b0dcd0a8bbb8507b3dbf2a2e0a2574ca022cd899f721c58dfa8a531ae1d4564f
Jul 17 18:13:51.161972 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:51.161938    2572 generic.go:356] "Generic (PLEG): container finished" podID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerID="a8a26f93e12873af8e31757b4e5e52bc560ae4d34250b874c2ba53fe219725bb" exitCode=6
Jul 17 18:13:51.162445 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:51.162017    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9" event={"ID":"b964fcbf-51a7-4ac1-9839-9fd11facdc90","Type":"ContainerDied","Data":"a8a26f93e12873af8e31757b4e5e52bc560ae4d34250b874c2ba53fe219725bb"}
Jul 17 18:13:51.162445 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:51.162099    2572 scope.go:117] "RemoveContainer" containerID="5c863443c8959b08021582a914afe4f82bde70d9f86878402bc8b4d7e81b90c7"
Jul 17 18:13:51.162445 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:51.162336    2572 scope.go:117] "RemoveContainer" containerID="a8a26f93e12873af8e31757b4e5e52bc560ae4d34250b874c2ba53fe219725bb"
Jul 17 18:13:51.162655 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:51.162632    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9_odh-ai-gateway-infra(b964fcbf-51a7-4ac1-9839-9fd11facdc90)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90"
Jul 17 18:13:51.163850 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:51.163825    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b" event={"ID":"09d419b7-2667-477c-85c0-de7f88bb23ee","Type":"ContainerStarted","Data":"95923c67b0017431b369a35d658bf032bdd5d8447b1292485dadd505def5e113"}
Jul 17 18:13:51.163992 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:51.163870    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b" event={"ID":"09d419b7-2667-477c-85c0-de7f88bb23ee","Type":"ContainerStarted","Data":"b0dcd0a8bbb8507b3dbf2a2e0a2574ca022cd899f721c58dfa8a531ae1d4564f"}
Jul 17 18:13:51.165625 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:51.165608    2572 scope.go:117] "RemoveContainer" containerID="e388d8e003bfb00356757aa8dca4b12c5eb7c5ee18e3d09a9cc20cb1386fad59"
Jul 17 18:13:51.165796 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:51.165776    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g_odh-ai-gateway-infra(f7d4315b-4fc4-4760-ac16-5706941e2b39)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39"
Jul 17 18:13:51.211785 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:51.211708    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b" podStartSLOduration=1.211692669 podStartE2EDuration="1.211692669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:13:51.20964707 +0000 UTC m=+3003.215002924" watchObservedRunningTime="2026-07-17 18:13:51.211692669 +0000 UTC m=+3003.217048523"
Jul 17 18:13:51.509688 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:51.509652    2572 scope.go:117] "RemoveContainer" containerID="1278db6d0e3d3ca0431b82b0041fcf2eae7ea04dd095ec9e11380285f037302e"
Jul 17 18:13:52.176982 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:52.176943    2572 generic.go:356] "Generic (PLEG): container finished" podID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerID="ac7591829f0c47cb892f8cfd1205264d40038e82b66fb485221fbde4f9fc83d5" exitCode=6
Jul 17 18:13:52.177477 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:52.177015    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5" event={"ID":"1c5cabdb-7cc4-4122-8233-15ff6d13fd32","Type":"ContainerDied","Data":"ac7591829f0c47cb892f8cfd1205264d40038e82b66fb485221fbde4f9fc83d5"}
Jul 17 18:13:52.177477 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:52.177079    2572 scope.go:117] "RemoveContainer" containerID="1278db6d0e3d3ca0431b82b0041fcf2eae7ea04dd095ec9e11380285f037302e"
Jul 17 18:13:52.177477 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:52.177450    2572 scope.go:117] "RemoveContainer" containerID="ac7591829f0c47cb892f8cfd1205264d40038e82b66fb485221fbde4f9fc83d5"
Jul 17 18:13:52.177766 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:52.177740    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5_odh-ai-gateway-infra(1c5cabdb-7cc4-4122-8233-15ff6d13fd32)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32"
Jul 17 18:13:52.179167 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:52.179145    2572 scope.go:117] "RemoveContainer" containerID="a8a26f93e12873af8e31757b4e5e52bc560ae4d34250b874c2ba53fe219725bb"
Jul 17 18:13:52.179427 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:13:52.179406    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9_odh-ai-gateway-infra(b964fcbf-51a7-4ac1-9839-9fd11facdc90)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90"
Jul 17 18:13:52.180461 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:52.180440    2572 generic.go:356] "Generic (PLEG): container finished" podID="09d419b7-2667-477c-85c0-de7f88bb23ee" containerID="95923c67b0017431b369a35d658bf032bdd5d8447b1292485dadd505def5e113" exitCode=0
Jul 17 18:13:52.180554 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:52.180531    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b" event={"ID":"09d419b7-2667-477c-85c0-de7f88bb23ee","Type":"ContainerDied","Data":"95923c67b0017431b369a35d658bf032bdd5d8447b1292485dadd505def5e113"}
Jul 17 18:13:53.203242 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:53.203210    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"]
Jul 17 18:13:53.327107 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:53.327076    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"
Jul 17 18:13:53.362159 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:53.362128    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"
Jul 17 18:13:53.404682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:53.404649    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5cabdb-7cc4-4122-8233-15ff6d13fd32-maas-api-service-ca\") pod \"1c5cabdb-7cc4-4122-8233-15ff6d13fd32\" (UID: \"1c5cabdb-7cc4-4122-8233-15ff6d13fd32\") "
Jul 17 18:13:53.404682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:53.404685    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d419b7-2667-477c-85c0-de7f88bb23ee-maas-api-service-ca\") pod \"09d419b7-2667-477c-85c0-de7f88bb23ee\" (UID: \"09d419b7-2667-477c-85c0-de7f88bb23ee\") "
Jul 17 18:13:53.405049 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:53.405026    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c5cabdb-7cc4-4122-8233-15ff6d13fd32-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1c5cabdb-7cc4-4122-8233-15ff6d13fd32" (UID: "1c5cabdb-7cc4-4122-8233-15ff6d13fd32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:13:53.405176 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:53.405151    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09d419b7-2667-477c-85c0-de7f88bb23ee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09d419b7-2667-477c-85c0-de7f88bb23ee" (UID: "09d419b7-2667-477c-85c0-de7f88bb23ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:13:53.505565 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:53.505429    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5cabdb-7cc4-4122-8233-15ff6d13fd32-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:13:53.505565 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:53.505464    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d419b7-2667-477c-85c0-de7f88bb23ee-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:13:54.196035 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:54.196001    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"
Jul 17 18:13:54.196246 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:54.196027    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5" event={"ID":"1c5cabdb-7cc4-4122-8233-15ff6d13fd32","Type":"ContainerDied","Data":"9c33f45dc214f0b1bf151db1d4e5c3265d654a0eb4bb5bb8a802aa7b9a7cf4d8"}
Jul 17 18:13:54.196246 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:54.196120    2572 scope.go:117] "RemoveContainer" containerID="ac7591829f0c47cb892f8cfd1205264d40038e82b66fb485221fbde4f9fc83d5"
Jul 17 18:13:54.197617 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:54.197597    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b" event={"ID":"09d419b7-2667-477c-85c0-de7f88bb23ee","Type":"ContainerDied","Data":"b0dcd0a8bbb8507b3dbf2a2e0a2574ca022cd899f721c58dfa8a531ae1d4564f"}
Jul 17 18:13:54.197617 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:54.197608    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b"
Jul 17 18:13:54.197765 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:54.197638    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0dcd0a8bbb8507b3dbf2a2e0a2574ca022cd899f721c58dfa8a531ae1d4564f"
Jul 17 18:13:54.231363 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:54.231325    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"]
Jul 17 18:13:54.233726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:54.233697    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nksx5"]
Jul 17 18:13:54.516138 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:13:54.516054    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" path="/var/lib/kubelet/pods/1c5cabdb-7cc4-4122-8233-15ff6d13fd32/volumes"
Jul 17 18:14:04.510243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:04.510206    2572 scope.go:117] "RemoveContainer" containerID="a8a26f93e12873af8e31757b4e5e52bc560ae4d34250b874c2ba53fe219725bb"
Jul 17 18:14:05.264537 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:05.264482    2572 generic.go:356] "Generic (PLEG): container finished" podID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerID="7b0a70da96671c05bde4fcc9b342ab8d160404d66ee1399b2ac41dafab78e97d" exitCode=6
Jul 17 18:14:05.264726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:05.264564    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9" event={"ID":"b964fcbf-51a7-4ac1-9839-9fd11facdc90","Type":"ContainerDied","Data":"7b0a70da96671c05bde4fcc9b342ab8d160404d66ee1399b2ac41dafab78e97d"}
Jul 17 18:14:05.264726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:05.264630    2572 scope.go:117] "RemoveContainer" containerID="a8a26f93e12873af8e31757b4e5e52bc560ae4d34250b874c2ba53fe219725bb"
Jul 17 18:14:05.264984 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:05.264963    2572 scope.go:117] "RemoveContainer" containerID="7b0a70da96671c05bde4fcc9b342ab8d160404d66ee1399b2ac41dafab78e97d"
Jul 17 18:14:05.265240 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:05.265221    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9_odh-ai-gateway-infra(b964fcbf-51a7-4ac1-9839-9fd11facdc90)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90"
Jul 17 18:14:05.509997 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:05.509903    2572 scope.go:117] "RemoveContainer" containerID="e388d8e003bfb00356757aa8dca4b12c5eb7c5ee18e3d09a9cc20cb1386fad59"
Jul 17 18:14:06.272145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:06.272108    2572 generic.go:356] "Generic (PLEG): container finished" podID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerID="8b90a61b105628319e04e037243cfb38031ef33bc4c0765fa06a8a5a31325e4c" exitCode=6
Jul 17 18:14:06.272145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:06.272137    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" event={"ID":"f7d4315b-4fc4-4760-ac16-5706941e2b39","Type":"ContainerDied","Data":"8b90a61b105628319e04e037243cfb38031ef33bc4c0765fa06a8a5a31325e4c"}
Jul 17 18:14:06.272662 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:06.272204    2572 scope.go:117] "RemoveContainer" containerID="e388d8e003bfb00356757aa8dca4b12c5eb7c5ee18e3d09a9cc20cb1386fad59"
Jul 17 18:14:06.272662 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:06.272529    2572 scope.go:117] "RemoveContainer" containerID="8b90a61b105628319e04e037243cfb38031ef33bc4c0765fa06a8a5a31325e4c"
Jul 17 18:14:06.272817 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:06.272796    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g_odh-ai-gateway-infra(f7d4315b-4fc4-4760-ac16-5706941e2b39)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39"
Jul 17 18:14:06.304696 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:06.304655    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"]
Jul 17 18:14:06.437366 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:06.437340    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"
Jul 17 18:14:06.518927 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:06.518897    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b964fcbf-51a7-4ac1-9839-9fd11facdc90-maas-api-service-ca\") pod \"b964fcbf-51a7-4ac1-9839-9fd11facdc90\" (UID: \"b964fcbf-51a7-4ac1-9839-9fd11facdc90\") "
Jul 17 18:14:06.519286 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:06.519261    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b964fcbf-51a7-4ac1-9839-9fd11facdc90-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b964fcbf-51a7-4ac1-9839-9fd11facdc90" (UID: "b964fcbf-51a7-4ac1-9839-9fd11facdc90"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:14:06.620405 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:06.620363    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b964fcbf-51a7-4ac1-9839-9fd11facdc90-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:14:07.281513 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.281458    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"
Jul 17 18:14:07.282032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.281509    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9" event={"ID":"b964fcbf-51a7-4ac1-9839-9fd11facdc90","Type":"ContainerDied","Data":"80d95701f94c6e795910c17a41dbd973fb1db8738c1d57a20baf49452cf93e76"}
Jul 17 18:14:07.282032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.281593    2572 scope.go:117] "RemoveContainer" containerID="7b0a70da96671c05bde4fcc9b342ab8d160404d66ee1399b2ac41dafab78e97d"
Jul 17 18:14:07.301915 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.301879    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"]
Jul 17 18:14:07.305765 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.305737    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"]
Jul 17 18:14:07.313934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.313905    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-trlw9"]
Jul 17 18:14:07.444216 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.444186    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"
Jul 17 18:14:07.528731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.528692    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d4315b-4fc4-4760-ac16-5706941e2b39-maas-api-service-ca\") pod \"f7d4315b-4fc4-4760-ac16-5706941e2b39\" (UID: \"f7d4315b-4fc4-4760-ac16-5706941e2b39\") "
Jul 17 18:14:07.529125 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.529095    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d4315b-4fc4-4760-ac16-5706941e2b39-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f7d4315b-4fc4-4760-ac16-5706941e2b39" (UID: "f7d4315b-4fc4-4760-ac16-5706941e2b39"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:14:07.630368 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:07.630323    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d4315b-4fc4-4760-ac16-5706941e2b39-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:14:08.292165 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:08.292130    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g" event={"ID":"f7d4315b-4fc4-4760-ac16-5706941e2b39","Type":"ContainerDied","Data":"6df6db4e5c0950b3b3f7f6a513ee15db6bcff09bd3e897e0e7c09b41f7817de2"}
Jul 17 18:14:08.292165 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:08.292156    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"
Jul 17 18:14:08.292721 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:08.292186    2572 scope.go:117] "RemoveContainer" containerID="8b90a61b105628319e04e037243cfb38031ef33bc4c0765fa06a8a5a31325e4c"
Jul 17 18:14:08.314362 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:08.314322    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"]
Jul 17 18:14:08.321862 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:08.321814    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hzc2g"]
Jul 17 18:14:08.515065 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:08.515030    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" path="/var/lib/kubelet/pods/b964fcbf-51a7-4ac1-9839-9fd11facdc90/volumes"
Jul 17 18:14:08.515403 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:08.515389    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" path="/var/lib/kubelet/pods/f7d4315b-4fc4-4760-ac16-5706941e2b39/volumes"
Jul 17 18:14:20.417815 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.417778    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"]
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418149    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418161    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418175    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418181    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418189    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418194    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418201    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418206    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418217    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418222    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418229    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418234    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418241    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09d419b7-2667-477c-85c0-de7f88bb23ee" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418247    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="09d419b7-2667-477c-85c0-de7f88bb23ee" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418302    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418310    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418316    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418323    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418331    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418335    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerName="revoke-keys"
Jul 17 18:14:20.418326 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418339    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerName="revoke-keys"
Jul 17 18:14:20.419050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.418345    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="09d419b7-2667-477c-85c0-de7f88bb23ee" containerName="revoke-keys"
Jul 17 18:14:20.423018 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.422989    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"
Jul 17 18:14:20.427832 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.427803    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:14:20.427832 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.427823    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:14:20.430655 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.430626    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"]
Jul 17 18:14:20.551055 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.551007    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d11ba2fb-2095-4c40-978e-fee74510772f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk\" (UID: \"d11ba2fb-2095-4c40-978e-fee74510772f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"
Jul 17 18:14:20.652197 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.652158    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d11ba2fb-2095-4c40-978e-fee74510772f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk\" (UID: \"d11ba2fb-2095-4c40-978e-fee74510772f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"
Jul 17 18:14:20.652811 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.652792    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d11ba2fb-2095-4c40-978e-fee74510772f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk\" (UID: \"d11ba2fb-2095-4c40-978e-fee74510772f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"
Jul 17 18:14:20.736117 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.736023    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"
Jul 17 18:14:20.881959 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:20.881928    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"]
Jul 17 18:14:20.883423 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:14:20.883396    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11ba2fb_2095_4c40_978e_fee74510772f.slice/crio-b8b4c6c010b31298f45689b78da592ae62f954d172d206fcd1649d74d48513e9 WatchSource:0}: Error finding container b8b4c6c010b31298f45689b78da592ae62f954d172d206fcd1649d74d48513e9: Status 404 returned error can't find the container with id b8b4c6c010b31298f45689b78da592ae62f954d172d206fcd1649d74d48513e9
Jul 17 18:14:21.367881 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:21.367843    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" event={"ID":"d11ba2fb-2095-4c40-978e-fee74510772f","Type":"ContainerStarted","Data":"cd9530837d693c7d25d9b9786d424e7843921122be88cdf6c8fea877aecec8a5"}
Jul 17 18:14:21.368093 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:21.367925    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" event={"ID":"d11ba2fb-2095-4c40-978e-fee74510772f","Type":"ContainerStarted","Data":"b8b4c6c010b31298f45689b78da592ae62f954d172d206fcd1649d74d48513e9"}
Jul 17 18:14:21.385856 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:21.385798    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" podStartSLOduration=1.385779876 podStartE2EDuration="1.385779876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:14:21.383892225 +0000 UTC m=+3033.389248091" watchObservedRunningTime="2026-07-17 18:14:21.385779876 +0000 UTC m=+3033.391135729"
Jul 17 18:14:22.374405 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:22.374364    2572 generic.go:356] "Generic (PLEG): container finished" podID="d11ba2fb-2095-4c40-978e-fee74510772f" containerID="cd9530837d693c7d25d9b9786d424e7843921122be88cdf6c8fea877aecec8a5" exitCode=6
Jul 17 18:14:22.374914 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:22.374429    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" event={"ID":"d11ba2fb-2095-4c40-978e-fee74510772f","Type":"ContainerDied","Data":"cd9530837d693c7d25d9b9786d424e7843921122be88cdf6c8fea877aecec8a5"}
Jul 17 18:14:22.374914 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:22.374678    2572 scope.go:117] "RemoveContainer" containerID="cd9530837d693c7d25d9b9786d424e7843921122be88cdf6c8fea877aecec8a5"
Jul 17 18:14:23.381951 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:23.381861    2572 generic.go:356] "Generic (PLEG): container finished" podID="d11ba2fb-2095-4c40-978e-fee74510772f" containerID="2c7e913f5ea04324a4a3dcff7c396d36c899604b9b26afcb445c672dda3be844" exitCode=6
Jul 17 18:14:23.382365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:23.381942    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" event={"ID":"d11ba2fb-2095-4c40-978e-fee74510772f","Type":"ContainerDied","Data":"2c7e913f5ea04324a4a3dcff7c396d36c899604b9b26afcb445c672dda3be844"}
Jul 17 18:14:23.382365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:23.382011    2572 scope.go:117] "RemoveContainer" containerID="cd9530837d693c7d25d9b9786d424e7843921122be88cdf6c8fea877aecec8a5"
Jul 17 18:14:23.382365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:23.382158    2572 scope.go:117] "RemoveContainer" containerID="2c7e913f5ea04324a4a3dcff7c396d36c899604b9b26afcb445c672dda3be844"
Jul 17 18:14:23.382579 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:23.382369    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk_odh-ai-gateway-infra(d11ba2fb-2095-4c40-978e-fee74510772f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" podUID="d11ba2fb-2095-4c40-978e-fee74510772f"
Jul 17 18:14:24.389054 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.389021    2572 scope.go:117] "RemoveContainer" containerID="2c7e913f5ea04324a4a3dcff7c396d36c899604b9b26afcb445c672dda3be844"
Jul 17 18:14:24.389538 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:24.389286    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk_odh-ai-gateway-infra(d11ba2fb-2095-4c40-978e-fee74510772f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" podUID="d11ba2fb-2095-4c40-978e-fee74510772f"
Jul 17 18:14:24.619707 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.619666    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"]
Jul 17 18:14:24.620006 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.619994    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerName="revoke-keys"
Jul 17 18:14:24.620050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.620007    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c5cabdb-7cc4-4122-8233-15ff6d13fd32" containerName="revoke-keys"
Jul 17 18:14:24.620050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.620020    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerName="revoke-keys"
Jul 17 18:14:24.620050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.620025    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerName="revoke-keys"
Jul 17 18:14:24.620050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.620041    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerName="revoke-keys"
Jul 17 18:14:24.620050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.620047    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerName="revoke-keys"
Jul 17 18:14:24.620206 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.620109    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b964fcbf-51a7-4ac1-9839-9fd11facdc90" containerName="revoke-keys"
Jul 17 18:14:24.620206 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.620119    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7d4315b-4fc4-4760-ac16-5706941e2b39" containerName="revoke-keys"
Jul 17 18:14:24.624444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.624420    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"
Jul 17 18:14:24.631387 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.631350    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"]
Jul 17 18:14:24.791680 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.791578    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c2f021-ba4f-403c-b941-5ba7a7bb03e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5\" (UID: \"89c2f021-ba4f-403c-b941-5ba7a7bb03e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"
Jul 17 18:14:24.892883 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.892830    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c2f021-ba4f-403c-b941-5ba7a7bb03e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5\" (UID: \"89c2f021-ba4f-403c-b941-5ba7a7bb03e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"
Jul 17 18:14:24.893575 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.893554    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c2f021-ba4f-403c-b941-5ba7a7bb03e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5\" (UID: \"89c2f021-ba4f-403c-b941-5ba7a7bb03e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"
Jul 17 18:14:24.937888 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:24.937837    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"
Jul 17 18:14:25.080334 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:25.080305    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"]
Jul 17 18:14:25.081526 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:14:25.081476    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c2f021_ba4f_403c_b941_5ba7a7bb03e1.slice/crio-694c8c5ef58a5c12868b9147d19f40a706497227115c97558c841946c5ac80c5 WatchSource:0}: Error finding container 694c8c5ef58a5c12868b9147d19f40a706497227115c97558c841946c5ac80c5: Status 404 returned error can't find the container with id 694c8c5ef58a5c12868b9147d19f40a706497227115c97558c841946c5ac80c5
Jul 17 18:14:25.396082 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:25.396045    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" event={"ID":"89c2f021-ba4f-403c-b941-5ba7a7bb03e1","Type":"ContainerStarted","Data":"88030c90adce230d6ba8510f9633cfb0404ecc4ebecf4ae7bde59daf1e1eb2fe"}
Jul 17 18:14:25.396544 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:25.396107    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" event={"ID":"89c2f021-ba4f-403c-b941-5ba7a7bb03e1","Type":"ContainerStarted","Data":"694c8c5ef58a5c12868b9147d19f40a706497227115c97558c841946c5ac80c5"}
Jul 17 18:14:25.413570 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:25.413511    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" podStartSLOduration=1.413470544 podStartE2EDuration="1.413470544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:14:25.412798983 +0000 UTC m=+3037.418154837" watchObservedRunningTime="2026-07-17 18:14:25.413470544 +0000 UTC m=+3037.418826399"
Jul 17 18:14:26.405016 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:26.404980    2572 generic.go:356] "Generic (PLEG): container finished" podID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerID="88030c90adce230d6ba8510f9633cfb0404ecc4ebecf4ae7bde59daf1e1eb2fe" exitCode=6
Jul 17 18:14:26.405411 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:26.405069    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" event={"ID":"89c2f021-ba4f-403c-b941-5ba7a7bb03e1","Type":"ContainerDied","Data":"88030c90adce230d6ba8510f9633cfb0404ecc4ebecf4ae7bde59daf1e1eb2fe"}
Jul 17 18:14:26.405411 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:26.405257    2572 scope.go:117] "RemoveContainer" containerID="88030c90adce230d6ba8510f9633cfb0404ecc4ebecf4ae7bde59daf1e1eb2fe"
Jul 17 18:14:27.413366 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:27.413323    2572 generic.go:356] "Generic (PLEG): container finished" podID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerID="765a864e168e0dcdc0897306b97b48206c8e798fa154895e7641e9bddb5da7a7" exitCode=6
Jul 17 18:14:27.413802 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:27.413401    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" event={"ID":"89c2f021-ba4f-403c-b941-5ba7a7bb03e1","Type":"ContainerDied","Data":"765a864e168e0dcdc0897306b97b48206c8e798fa154895e7641e9bddb5da7a7"}
Jul 17 18:14:27.413802 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:27.413467    2572 scope.go:117] "RemoveContainer" containerID="88030c90adce230d6ba8510f9633cfb0404ecc4ebecf4ae7bde59daf1e1eb2fe"
Jul 17 18:14:27.413802 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:27.413719    2572 scope.go:117] "RemoveContainer" containerID="765a864e168e0dcdc0897306b97b48206c8e798fa154895e7641e9bddb5da7a7"
Jul 17 18:14:27.413992 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:27.413972    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5_odh-ai-gateway-infra(89c2f021-ba4f-403c-b941-5ba7a7bb03e1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1"
Jul 17 18:14:28.420725 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:28.420701    2572 scope.go:117] "RemoveContainer" containerID="765a864e168e0dcdc0897306b97b48206c8e798fa154895e7641e9bddb5da7a7"
Jul 17 18:14:28.421112 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:28.420909    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5_odh-ai-gateway-infra(89c2f021-ba4f-403c-b941-5ba7a7bb03e1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1"
Jul 17 18:14:38.785815 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:38.785774    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"]
Jul 17 18:14:38.789247 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:38.789225    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"
Jul 17 18:14:38.798610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:38.798569    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"]
Jul 17 18:14:38.920063 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:38.920021    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2fd6322-e8e2-46cf-9e10-994ce19d79fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv\" (UID: \"e2fd6322-e8e2-46cf-9e10-994ce19d79fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"
Jul 17 18:14:39.021291 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:39.021229    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2fd6322-e8e2-46cf-9e10-994ce19d79fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv\" (UID: \"e2fd6322-e8e2-46cf-9e10-994ce19d79fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"
Jul 17 18:14:39.021919 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:39.021899    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2fd6322-e8e2-46cf-9e10-994ce19d79fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv\" (UID: \"e2fd6322-e8e2-46cf-9e10-994ce19d79fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"
Jul 17 18:14:39.100990 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:39.100953    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"
Jul 17 18:14:39.244023 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:39.243983    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"]
Jul 17 18:14:39.245621 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:14:39.245590    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fd6322_e8e2_46cf_9e10_994ce19d79fa.slice/crio-96889f69c07bb25656e030bfac4963ca712ee88a395a9a58bf321763ea1ad702 WatchSource:0}: Error finding container 96889f69c07bb25656e030bfac4963ca712ee88a395a9a58bf321763ea1ad702: Status 404 returned error can't find the container with id 96889f69c07bb25656e030bfac4963ca712ee88a395a9a58bf321763ea1ad702
Jul 17 18:14:39.483385 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:39.483344    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv" event={"ID":"e2fd6322-e8e2-46cf-9e10-994ce19d79fa","Type":"ContainerStarted","Data":"96889f69c07bb25656e030bfac4963ca712ee88a395a9a58bf321763ea1ad702"}
Jul 17 18:14:39.509918 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:39.509884    2572 scope.go:117] "RemoveContainer" containerID="2c7e913f5ea04324a4a3dcff7c396d36c899604b9b26afcb445c672dda3be844"
Jul 17 18:14:40.490354 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:40.490224    2572 generic.go:356] "Generic (PLEG): container finished" podID="d11ba2fb-2095-4c40-978e-fee74510772f" containerID="e9fc4b9550cf817377d5307bb8fa1bff3b45c100aa9d4a74b5219a2022374316" exitCode=6
Jul 17 18:14:40.490354 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:40.490305    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" event={"ID":"d11ba2fb-2095-4c40-978e-fee74510772f","Type":"ContainerDied","Data":"e9fc4b9550cf817377d5307bb8fa1bff3b45c100aa9d4a74b5219a2022374316"}
Jul 17 18:14:40.490903 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:40.490366    2572 scope.go:117] "RemoveContainer" containerID="2c7e913f5ea04324a4a3dcff7c396d36c899604b9b26afcb445c672dda3be844"
Jul 17 18:14:40.490903 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:40.490740    2572 scope.go:117] "RemoveContainer" containerID="e9fc4b9550cf817377d5307bb8fa1bff3b45c100aa9d4a74b5219a2022374316"
Jul 17 18:14:40.491008 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:40.490973    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk_odh-ai-gateway-infra(d11ba2fb-2095-4c40-978e-fee74510772f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" podUID="d11ba2fb-2095-4c40-978e-fee74510772f"
Jul 17 18:14:40.492115 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:40.492091    2572 generic.go:356] "Generic (PLEG): container finished" podID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerID="d1db0ded0c7a1db24418cb3fdecbf46d124a116fe47f8ceffd71904a3f518783" exitCode=6
Jul 17 18:14:40.492226 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:40.492167    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv" event={"ID":"e2fd6322-e8e2-46cf-9e10-994ce19d79fa","Type":"ContainerDied","Data":"d1db0ded0c7a1db24418cb3fdecbf46d124a116fe47f8ceffd71904a3f518783"}
Jul 17 18:14:40.492313 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:40.492299    2572 scope.go:117] "RemoveContainer" containerID="d1db0ded0c7a1db24418cb3fdecbf46d124a116fe47f8ceffd71904a3f518783"
Jul 17 18:14:41.501591 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.501470    2572 generic.go:356] "Generic (PLEG): container finished" podID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerID="c8edb39b473deee9ecbb7a8a843e0e10b8f7e934cc0d6cade3dc771b8a3fecb9" exitCode=6
Jul 17 18:14:41.501591 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.501533    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv" event={"ID":"e2fd6322-e8e2-46cf-9e10-994ce19d79fa","Type":"ContainerDied","Data":"c8edb39b473deee9ecbb7a8a843e0e10b8f7e934cc0d6cade3dc771b8a3fecb9"}
Jul 17 18:14:41.502077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.501606    2572 scope.go:117] "RemoveContainer" containerID="d1db0ded0c7a1db24418cb3fdecbf46d124a116fe47f8ceffd71904a3f518783"
Jul 17 18:14:41.502077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.501815    2572 scope.go:117] "RemoveContainer" containerID="c8edb39b473deee9ecbb7a8a843e0e10b8f7e934cc0d6cade3dc771b8a3fecb9"
Jul 17 18:14:41.502077 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:41.502017    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv_odh-ai-gateway-infra(e2fd6322-e8e2-46cf-9e10-994ce19d79fa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa"
Jul 17 18:14:41.509868 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.509847    2572 scope.go:117] "RemoveContainer" containerID="765a864e168e0dcdc0897306b97b48206c8e798fa154895e7641e9bddb5da7a7"
Jul 17 18:14:41.521079 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.521050    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"]
Jul 17 18:14:41.666114 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.666084    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"
Jul 17 18:14:41.744593 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.744561    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d11ba2fb-2095-4c40-978e-fee74510772f-maas-api-service-ca\") pod \"d11ba2fb-2095-4c40-978e-fee74510772f\" (UID: \"d11ba2fb-2095-4c40-978e-fee74510772f\") "
Jul 17 18:14:41.744905 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.744886    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d11ba2fb-2095-4c40-978e-fee74510772f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d11ba2fb-2095-4c40-978e-fee74510772f" (UID: "d11ba2fb-2095-4c40-978e-fee74510772f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:14:41.845475 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:41.845447    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d11ba2fb-2095-4c40-978e-fee74510772f-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:14:42.515159 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.515066    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"
Jul 17 18:14:42.516839 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.516808    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk" event={"ID":"d11ba2fb-2095-4c40-978e-fee74510772f","Type":"ContainerDied","Data":"b8b4c6c010b31298f45689b78da592ae62f954d172d206fcd1649d74d48513e9"}
Jul 17 18:14:42.516991 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.516864    2572 scope.go:117] "RemoveContainer" containerID="e9fc4b9550cf817377d5307bb8fa1bff3b45c100aa9d4a74b5219a2022374316"
Jul 17 18:14:42.517060 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.517027    2572 generic.go:356] "Generic (PLEG): container finished" podID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerID="360ff4614cdfefa65b3fcb53e8c194e24bc9f34d54d2b419ea0dee253ac8437e" exitCode=6
Jul 17 18:14:42.517130 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.517105    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" event={"ID":"89c2f021-ba4f-403c-b941-5ba7a7bb03e1","Type":"ContainerDied","Data":"360ff4614cdfefa65b3fcb53e8c194e24bc9f34d54d2b419ea0dee253ac8437e"}
Jul 17 18:14:42.517433 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.517412    2572 scope.go:117] "RemoveContainer" containerID="360ff4614cdfefa65b3fcb53e8c194e24bc9f34d54d2b419ea0dee253ac8437e"
Jul 17 18:14:42.517700 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:42.517674    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5_odh-ai-gateway-infra(89c2f021-ba4f-403c-b941-5ba7a7bb03e1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1"
Jul 17 18:14:42.519185 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.519168    2572 scope.go:117] "RemoveContainer" containerID="c8edb39b473deee9ecbb7a8a843e0e10b8f7e934cc0d6cade3dc771b8a3fecb9"
Jul 17 18:14:42.519369 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:42.519350    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv_odh-ai-gateway-infra(e2fd6322-e8e2-46cf-9e10-994ce19d79fa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa"
Jul 17 18:14:42.528624 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.528600    2572 scope.go:117] "RemoveContainer" containerID="765a864e168e0dcdc0897306b97b48206c8e798fa154895e7641e9bddb5da7a7"
Jul 17 18:14:42.548128 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.548089    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"]
Jul 17 18:14:42.552471 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:42.552433    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-qq5vk"]
Jul 17 18:14:43.545681 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:43.545640    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"]
Jul 17 18:14:43.682656 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:43.682630    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"
Jul 17 18:14:43.761880 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:43.761846    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c2f021-ba4f-403c-b941-5ba7a7bb03e1-maas-api-service-ca\") pod \"89c2f021-ba4f-403c-b941-5ba7a7bb03e1\" (UID: \"89c2f021-ba4f-403c-b941-5ba7a7bb03e1\") "
Jul 17 18:14:43.762230 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:43.762204    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89c2f021-ba4f-403c-b941-5ba7a7bb03e1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89c2f021-ba4f-403c-b941-5ba7a7bb03e1" (UID: "89c2f021-ba4f-403c-b941-5ba7a7bb03e1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:14:43.862953 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:43.862917    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c2f021-ba4f-403c-b941-5ba7a7bb03e1-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:14:44.515524 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:44.515475    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" path="/var/lib/kubelet/pods/d11ba2fb-2095-4c40-978e-fee74510772f/volumes"
Jul 17 18:14:44.534993 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:44.534965    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"
Jul 17 18:14:44.535178 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:44.535009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5" event={"ID":"89c2f021-ba4f-403c-b941-5ba7a7bb03e1","Type":"ContainerDied","Data":"694c8c5ef58a5c12868b9147d19f40a706497227115c97558c841946c5ac80c5"}
Jul 17 18:14:44.535178 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:44.535067    2572 scope.go:117] "RemoveContainer" containerID="360ff4614cdfefa65b3fcb53e8c194e24bc9f34d54d2b419ea0dee253ac8437e"
Jul 17 18:14:44.562445 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:44.562407    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"]
Jul 17 18:14:44.570474 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:44.570442    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-9f5v5"]
Jul 17 18:14:46.515921 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.515874    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" path="/var/lib/kubelet/pods/89c2f021-ba4f-403c-b941-5ba7a7bb03e1/volumes"
Jul 17 18:14:46.876789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.876748    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"]
Jul 17 18:14:46.877126 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877114    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerName="revoke-keys"
Jul 17 18:14:46.877191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877127    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerName="revoke-keys"
Jul 17 18:14:46.877191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877146    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" containerName="revoke-keys"
Jul 17 18:14:46.877191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877151    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" containerName="revoke-keys"
Jul 17 18:14:46.877191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877158    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerName="revoke-keys"
Jul 17 18:14:46.877191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877164    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerName="revoke-keys"
Jul 17 18:14:46.877191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877172    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" containerName="revoke-keys"
Jul 17 18:14:46.877191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877177    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" containerName="revoke-keys"
Jul 17 18:14:46.877394 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877234    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" containerName="revoke-keys"
Jul 17 18:14:46.877394 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877240    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerName="revoke-keys"
Jul 17 18:14:46.877394 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877246    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" containerName="revoke-keys"
Jul 17 18:14:46.877394 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.877251    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerName="revoke-keys"
Jul 17 18:14:46.881789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.881768    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"
Jul 17 18:14:46.884300 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.884265    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8735b367-d456-423c-8733-de45311c1a42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4\" (UID: \"8735b367-d456-423c-8733-de45311c1a42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"
Jul 17 18:14:46.891198 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.891170    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"]
Jul 17 18:14:46.985645 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.985589    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8735b367-d456-423c-8733-de45311c1a42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4\" (UID: \"8735b367-d456-423c-8733-de45311c1a42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"
Jul 17 18:14:46.986235 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:46.986211    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8735b367-d456-423c-8733-de45311c1a42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4\" (UID: \"8735b367-d456-423c-8733-de45311c1a42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"
Jul 17 18:14:47.193986 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:47.193863    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"
Jul 17 18:14:47.363286 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:47.363255    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"]
Jul 17 18:14:47.365027 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:14:47.364996    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8735b367_d456_423c_8733_de45311c1a42.slice/crio-f03000bf34958dad23fe5aed7535fbb660ad8604216ce6a7a2175a86c44bef18 WatchSource:0}: Error finding container f03000bf34958dad23fe5aed7535fbb660ad8604216ce6a7a2175a86c44bef18: Status 404 returned error can't find the container with id f03000bf34958dad23fe5aed7535fbb660ad8604216ce6a7a2175a86c44bef18
Jul 17 18:14:47.555412 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:47.555369    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4" event={"ID":"8735b367-d456-423c-8733-de45311c1a42","Type":"ContainerStarted","Data":"f03000bf34958dad23fe5aed7535fbb660ad8604216ce6a7a2175a86c44bef18"}
Jul 17 18:14:48.562448 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:48.562414    2572 generic.go:356] "Generic (PLEG): container finished" podID="8735b367-d456-423c-8733-de45311c1a42" containerID="ee5ccec0ad9d2e8d9f486c7d00e5c16356368a4ae066b062e1807f65dcd557a4" exitCode=6
Jul 17 18:14:48.562962 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:48.562519    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4" event={"ID":"8735b367-d456-423c-8733-de45311c1a42","Type":"ContainerDied","Data":"ee5ccec0ad9d2e8d9f486c7d00e5c16356368a4ae066b062e1807f65dcd557a4"}
Jul 17 18:14:48.562962 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:48.562761    2572 scope.go:117] "RemoveContainer" containerID="ee5ccec0ad9d2e8d9f486c7d00e5c16356368a4ae066b062e1807f65dcd557a4"
Jul 17 18:14:49.570159 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:49.570120    2572 generic.go:356] "Generic (PLEG): container finished" podID="8735b367-d456-423c-8733-de45311c1a42" containerID="0db57f7ddbda90f671d87d837dc6250e95d06b81326ee45ebfd28a32a4827691" exitCode=6
Jul 17 18:14:49.570670 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:49.570180    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4" event={"ID":"8735b367-d456-423c-8733-de45311c1a42","Type":"ContainerDied","Data":"0db57f7ddbda90f671d87d837dc6250e95d06b81326ee45ebfd28a32a4827691"}
Jul 17 18:14:49.570670 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:49.570264    2572 scope.go:117] "RemoveContainer" containerID="ee5ccec0ad9d2e8d9f486c7d00e5c16356368a4ae066b062e1807f65dcd557a4"
Jul 17 18:14:49.570670 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:49.570457    2572 scope.go:117] "RemoveContainer" containerID="0db57f7ddbda90f671d87d837dc6250e95d06b81326ee45ebfd28a32a4827691"
Jul 17 18:14:49.570807 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:49.570739    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4_odh-ai-gateway-infra(8735b367-d456-423c-8733-de45311c1a42)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4" podUID="8735b367-d456-423c-8733-de45311c1a42"
Jul 17 18:14:50.577414 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:50.577387    2572 scope.go:117] "RemoveContainer" containerID="0db57f7ddbda90f671d87d837dc6250e95d06b81326ee45ebfd28a32a4827691"
Jul 17 18:14:50.577839 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:50.577591    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4_odh-ai-gateway-infra(8735b367-d456-423c-8733-de45311c1a42)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4" podUID="8735b367-d456-423c-8733-de45311c1a42"
Jul 17 18:14:54.510682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:54.510640    2572 scope.go:117] "RemoveContainer" containerID="c8edb39b473deee9ecbb7a8a843e0e10b8f7e934cc0d6cade3dc771b8a3fecb9"
Jul 17 18:14:55.607533 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:55.607473    2572 generic.go:356] "Generic (PLEG): container finished" podID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerID="67c7b5c7ca72fbfcf73168a4f5ca5259790b5b6bb1fd490250614be9789f0699" exitCode=6
Jul 17 18:14:55.608007 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:55.607549    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv" event={"ID":"e2fd6322-e8e2-46cf-9e10-994ce19d79fa","Type":"ContainerDied","Data":"67c7b5c7ca72fbfcf73168a4f5ca5259790b5b6bb1fd490250614be9789f0699"}
Jul 17 18:14:55.608007 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:55.607613    2572 scope.go:117] "RemoveContainer" containerID="c8edb39b473deee9ecbb7a8a843e0e10b8f7e934cc0d6cade3dc771b8a3fecb9"
Jul 17 18:14:55.608007 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:55.607892    2572 scope.go:117] "RemoveContainer" containerID="67c7b5c7ca72fbfcf73168a4f5ca5259790b5b6bb1fd490250614be9789f0699"
Jul 17 18:14:55.608196 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:14:55.608129    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv_odh-ai-gateway-infra(e2fd6322-e8e2-46cf-9e10-994ce19d79fa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa"
Jul 17 18:14:56.636806 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:56.636761    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"]
Jul 17 18:14:56.779539 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:56.779505    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"
Jul 17 18:14:56.858202 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:56.858167    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2fd6322-e8e2-46cf-9e10-994ce19d79fa-maas-api-service-ca\") pod \"e2fd6322-e8e2-46cf-9e10-994ce19d79fa\" (UID: \"e2fd6322-e8e2-46cf-9e10-994ce19d79fa\") "
Jul 17 18:14:56.858593 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:56.858570    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2fd6322-e8e2-46cf-9e10-994ce19d79fa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2fd6322-e8e2-46cf-9e10-994ce19d79fa" (UID: "e2fd6322-e8e2-46cf-9e10-994ce19d79fa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:14:56.959562 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:56.959463    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2fd6322-e8e2-46cf-9e10-994ce19d79fa-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:14:57.623220 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:57.623181    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"
Jul 17 18:14:57.623220 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:57.623201    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv" event={"ID":"e2fd6322-e8e2-46cf-9e10-994ce19d79fa","Type":"ContainerDied","Data":"96889f69c07bb25656e030bfac4963ca712ee88a395a9a58bf321763ea1ad702"}
Jul 17 18:14:57.623525 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:57.623277    2572 scope.go:117] "RemoveContainer" containerID="67c7b5c7ca72fbfcf73168a4f5ca5259790b5b6bb1fd490250614be9789f0699"
Jul 17 18:14:57.660474 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:57.660435    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"]
Jul 17 18:14:57.668737 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:57.668689    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-dxlpv"]
Jul 17 18:14:58.515758 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:14:58.515716    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" path="/var/lib/kubelet/pods/e2fd6322-e8e2-46cf-9e10-994ce19d79fa/volumes"
Jul 17 18:15:00.131588 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131549    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p"]
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131917    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerName="revoke-keys"
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131929    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerName="revoke-keys"
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131938    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" containerName="revoke-keys"
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131944    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" containerName="revoke-keys"
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131954    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerName="revoke-keys"
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131960    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerName="revoke-keys"
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131973    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerName="revoke-keys"
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131981    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerName="revoke-keys"
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.131990    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerName="revoke-keys"
Jul 17 18:15:00.132020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.132023    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerName="revoke-keys"
Jul 17 18:15:00.132597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.132103    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c2f021-ba4f-403c-b941-5ba7a7bb03e1" containerName="revoke-keys"
Jul 17 18:15:00.132597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.132114    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerName="revoke-keys"
Jul 17 18:15:00.132597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.132120    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerName="revoke-keys"
Jul 17 18:15:00.132597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.132126    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fd6322-e8e2-46cf-9e10-994ce19d79fa" containerName="revoke-keys"
Jul 17 18:15:00.132597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.132131    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d11ba2fb-2095-4c40-978e-fee74510772f" containerName="revoke-keys"
Jul 17 18:15:00.135167 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.135143    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p"
Jul 17 18:15:00.135318 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.135163    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx"]
Jul 17 18:15:00.138543 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.138508    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59"]
Jul 17 18:15:00.138665 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.138585    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx"
Jul 17 18:15:00.143210 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.143185    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59"
Jul 17 18:15:00.146429 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.146368    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p"]
Jul 17 18:15:00.149666 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.149383    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p"
Jul 17 18:15:00.149940 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.149913    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59"]
Jul 17 18:15:00.159391 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.159363    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx"
Jul 17 18:15:00.165626 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.165594    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx"]
Jul 17 18:15:00.195539 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.195509    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59"
Jul 17 18:15:00.372515 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.372477    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx"]
Jul 17 18:15:00.373770 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:15:00.373739    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ddc285_dad8_482c_986b_0553b9b70828.slice/crio-8e439dbd8ea1aa52b4f397760bbc1bb4c77a63ba89374ca417d4b1b871057ea5 WatchSource:0}: Error finding container 8e439dbd8ea1aa52b4f397760bbc1bb4c77a63ba89374ca417d4b1b871057ea5: Status 404 returned error can't find the container with id 8e439dbd8ea1aa52b4f397760bbc1bb4c77a63ba89374ca417d4b1b871057ea5
Jul 17 18:15:00.382241 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.382217    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59"]
Jul 17 18:15:00.387007 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:15:00.386976    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a94db1_06f4_435e_ae49_38df53761038.slice/crio-057710aab52b2c2e48e3d5bb527fc61430c76a69a0ca9b0ffe4891ec6cdb15ec WatchSource:0}: Error finding container 057710aab52b2c2e48e3d5bb527fc61430c76a69a0ca9b0ffe4891ec6cdb15ec: Status 404 returned error can't find the container with id 057710aab52b2c2e48e3d5bb527fc61430c76a69a0ca9b0ffe4891ec6cdb15ec
Jul 17 18:15:00.528599 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.528571    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p"]
Jul 17 18:15:00.544833 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:15:00.544796    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffeeee75_61c9_46fc_9cc3_353864bc8fb0.slice/crio-fb93fcb20edcfd1967e5c5ebebfbeea26d21ae5792c6cc5260d9c98a5072efd6 WatchSource:0}: Error finding container fb93fcb20edcfd1967e5c5ebebfbeea26d21ae5792c6cc5260d9c98a5072efd6: Status 404 returned error can't find the container with id fb93fcb20edcfd1967e5c5ebebfbeea26d21ae5792c6cc5260d9c98a5072efd6
Jul 17 18:15:00.647636 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.647602    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx" event={"ID":"03ddc285-dad8-482c-986b-0553b9b70828","Type":"ContainerStarted","Data":"b79b229052d76ed9927fc81bff22ec99d1a84f1c66d17af15ce2416b22ab076a"}
Jul 17 18:15:00.647841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.647671    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx" event={"ID":"03ddc285-dad8-482c-986b-0553b9b70828","Type":"ContainerStarted","Data":"8e439dbd8ea1aa52b4f397760bbc1bb4c77a63ba89374ca417d4b1b871057ea5"}
Jul 17 18:15:00.648845 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.648813    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p" event={"ID":"ffeeee75-61c9-46fc-9cc3-353864bc8fb0","Type":"ContainerStarted","Data":"fb93fcb20edcfd1967e5c5ebebfbeea26d21ae5792c6cc5260d9c98a5072efd6"}
Jul 17 18:15:00.650348 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.650318    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59" event={"ID":"06a94db1-06f4-435e-ae49-38df53761038","Type":"ContainerStarted","Data":"1ea969402a2745139a6e91988ad11a55b7054ba354f7825b56b95cff07217014"}
Jul 17 18:15:00.650460 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.650364    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59" event={"ID":"06a94db1-06f4-435e-ae49-38df53761038","Type":"ContainerStarted","Data":"057710aab52b2c2e48e3d5bb527fc61430c76a69a0ca9b0ffe4891ec6cdb15ec"}
Jul 17 18:15:00.664752 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.664695    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx" podStartSLOduration=0.664678568 podStartE2EDuration="664.678568ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:15:00.662208877 +0000 UTC m=+3072.667564730" watchObservedRunningTime="2026-07-17 18:15:00.664678568 +0000 UTC m=+3072.670034421"
Jul 17 18:15:00.679961 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:00.679893    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59" podStartSLOduration=0.679874412 podStartE2EDuration="679.874412ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:15:00.677234868 +0000 UTC m=+3072.682590724" watchObservedRunningTime="2026-07-17 18:15:00.679874412 +0000 UTC m=+3072.685230264"
Jul 17 18:15:01.657087 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:01.657052    2572 generic.go:356] "Generic (PLEG): container finished" podID="03ddc285-dad8-482c-986b-0553b9b70828" containerID="b79b229052d76ed9927fc81bff22ec99d1a84f1c66d17af15ce2416b22ab076a" exitCode=0
Jul 17 18:15:01.657596 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:01.657136    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx" event={"ID":"03ddc285-dad8-482c-986b-0553b9b70828","Type":"ContainerDied","Data":"b79b229052d76ed9927fc81bff22ec99d1a84f1c66d17af15ce2416b22ab076a"}
Jul 17 18:15:01.658815 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:01.658788    2572 generic.go:356] "Generic (PLEG): container finished" podID="ffeeee75-61c9-46fc-9cc3-353864bc8fb0" containerID="06095cbb59c5cfba1abaaf5ec6ffd3a3576311dc7c60f6fced17d5c0f733eeeb" exitCode=0
Jul 17 18:15:01.658925 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:01.658871    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p" event={"ID":"ffeeee75-61c9-46fc-9cc3-353864bc8fb0","Type":"ContainerDied","Data":"06095cbb59c5cfba1abaaf5ec6ffd3a3576311dc7c60f6fced17d5c0f733eeeb"}
Jul 17 18:15:01.660513 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:01.660472    2572 generic.go:356] "Generic (PLEG): container finished" podID="06a94db1-06f4-435e-ae49-38df53761038" containerID="1ea969402a2745139a6e91988ad11a55b7054ba354f7825b56b95cff07217014" exitCode=0
Jul 17 18:15:01.660652 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:01.660520    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59" event={"ID":"06a94db1-06f4-435e-ae49-38df53761038","Type":"ContainerDied","Data":"1ea969402a2745139a6e91988ad11a55b7054ba354f7825b56b95cff07217014"}
Jul 17 18:15:02.817614 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:02.817581    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p"
Jul 17 18:15:02.838397 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:02.838363    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59"
Jul 17 18:15:02.872969 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:02.872938    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx"
Jul 17 18:15:03.676002 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:03.675973    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx"
Jul 17 18:15:03.676183 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:03.675972    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-lkngx" event={"ID":"03ddc285-dad8-482c-986b-0553b9b70828","Type":"ContainerDied","Data":"8e439dbd8ea1aa52b4f397760bbc1bb4c77a63ba89374ca417d4b1b871057ea5"}
Jul 17 18:15:03.676183 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:03.676099    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e439dbd8ea1aa52b4f397760bbc1bb4c77a63ba89374ca417d4b1b871057ea5"
Jul 17 18:15:03.677340 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:03.677323    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p"
Jul 17 18:15:03.677413 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:03.677347    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p" event={"ID":"ffeeee75-61c9-46fc-9cc3-353864bc8fb0","Type":"ContainerDied","Data":"fb93fcb20edcfd1967e5c5ebebfbeea26d21ae5792c6cc5260d9c98a5072efd6"}
Jul 17 18:15:03.677413 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:03.677387    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb93fcb20edcfd1967e5c5ebebfbeea26d21ae5792c6cc5260d9c98a5072efd6"
Jul 17 18:15:03.678759 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:03.678733    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59"
Jul 17 18:15:03.678759 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:03.678746    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59" event={"ID":"06a94db1-06f4-435e-ae49-38df53761038","Type":"ContainerDied","Data":"057710aab52b2c2e48e3d5bb527fc61430c76a69a0ca9b0ffe4891ec6cdb15ec"}
Jul 17 18:15:03.678909 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:03.678783    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="057710aab52b2c2e48e3d5bb527fc61430c76a69a0ca9b0ffe4891ec6cdb15ec"
Jul 17 18:15:05.509928 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:05.509897    2572 scope.go:117] "RemoveContainer" containerID="0db57f7ddbda90f671d87d837dc6250e95d06b81326ee45ebfd28a32a4827691"
Jul 17 18:15:06.697263 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:06.697229    2572 generic.go:356] "Generic (PLEG): container finished" podID="8735b367-d456-423c-8733-de45311c1a42" containerID="3be7d8c967efd81f3bd922a83140bfd9590197f92d8fd6fb51a6e15d5c991dba" exitCode=6
Jul 17 18:15:06.697729 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:06.697305    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4" event={"ID":"8735b367-d456-423c-8733-de45311c1a42","Type":"ContainerDied","Data":"3be7d8c967efd81f3bd922a83140bfd9590197f92d8fd6fb51a6e15d5c991dba"}
Jul 17 18:15:06.697729 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:06.697369    2572 scope.go:117] "RemoveContainer" containerID="0db57f7ddbda90f671d87d837dc6250e95d06b81326ee45ebfd28a32a4827691"
Jul 17 18:15:06.697836 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:06.697726    2572 scope.go:117] "RemoveContainer" containerID="3be7d8c967efd81f3bd922a83140bfd9590197f92d8fd6fb51a6e15d5c991dba"
Jul 17 18:15:06.697944 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:06.697922    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4_odh-ai-gateway-infra(8735b367-d456-423c-8733-de45311c1a42)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4" podUID="8735b367-d456-423c-8733-de45311c1a42"
Jul 17 18:15:07.287331 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287300    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"]
Jul 17 18:15:07.287679 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287663    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffeeee75-61c9-46fc-9cc3-353864bc8fb0" containerName="cleanup"
Jul 17 18:15:07.287737 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287682    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffeeee75-61c9-46fc-9cc3-353864bc8fb0" containerName="cleanup"
Jul 17 18:15:07.287780 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287740    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06a94db1-06f4-435e-ae49-38df53761038" containerName="cleanup"
Jul 17 18:15:07.287780 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287753    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="06a94db1-06f4-435e-ae49-38df53761038" containerName="cleanup"
Jul 17 18:15:07.287780 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287774    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03ddc285-dad8-482c-986b-0553b9b70828" containerName="cleanup"
Jul 17 18:15:07.287873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287785    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="03ddc285-dad8-482c-986b-0553b9b70828" containerName="cleanup"
Jul 17 18:15:07.287906 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287875    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="03ddc285-dad8-482c-986b-0553b9b70828" containerName="cleanup"
Jul 17 18:15:07.287906 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287883    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="06a94db1-06f4-435e-ae49-38df53761038" containerName="cleanup"
Jul 17 18:15:07.287906 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.287892    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffeeee75-61c9-46fc-9cc3-353864bc8fb0" containerName="cleanup"
Jul 17 18:15:07.292039 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.292015    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.294960 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.294931    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-22fb10\"/\"kube-root-ca.crt\""
Jul 17 18:15:07.296204 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.296175    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-22fb10\"/\"auth-test-model-22fb10-kserve-self-signed-certs\""
Jul 17 18:15:07.296677 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.296390    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-22fb10\"/\"default-dockercfg-qw92v\""
Jul 17 18:15:07.296677 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.296429    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-22fb10\"/\"openshift-service-ca.crt\""
Jul 17 18:15:07.300716 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.300686    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"]
Jul 17 18:15:07.344295 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.344257    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tmp-dir\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.344295 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.344299    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-home\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.344622 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.344350    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8gsg\" (UniqueName: \"kubernetes.io/projected/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-kube-api-access-r8gsg\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.344622 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.344379    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-dshm\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.344622 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.344403    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-model-cache\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.344622 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.344523    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tls-certs\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.445572 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.445530    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8gsg\" (UniqueName: \"kubernetes.io/projected/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-kube-api-access-r8gsg\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.445749 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.445579    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-dshm\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.445749 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.445695    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-model-cache\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.445825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.445777    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tls-certs\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.445825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.445817    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tmp-dir\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.445922 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.445844    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-home\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.446098 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.446075    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-model-cache\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.446183 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.446157    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-home\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.446247 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.446208    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tmp-dir\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.448793 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.448285    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-dshm\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.448934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.448840    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tls-certs\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.454026 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.453999    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8gsg\" (UniqueName: \"kubernetes.io/projected/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-kube-api-access-r8gsg\") pod \"auth-test-model-22fb10-kserve-84994fd847-d5472\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") " pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.603314 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.603275    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:07.724987 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.724946    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"]
Jul 17 18:15:07.749534 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.749341    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"]
Jul 17 18:15:07.752552 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:15:07.752519    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49b8ba94_5cc8_4a8d_b7a4_a72aad654786.slice/crio-7a7e386029646b9a84c680c2cd609da8e3dc83909f2db7bbdd6d1f85b60ac01f WatchSource:0}: Error finding container 7a7e386029646b9a84c680c2cd609da8e3dc83909f2db7bbdd6d1f85b60ac01f: Status 404 returned error can't find the container with id 7a7e386029646b9a84c680c2cd609da8e3dc83909f2db7bbdd6d1f85b60ac01f
Jul 17 18:15:07.862295 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.862264    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"
Jul 17 18:15:07.951058 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.951022    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8735b367-d456-423c-8733-de45311c1a42-maas-api-service-ca\") pod \"8735b367-d456-423c-8733-de45311c1a42\" (UID: \"8735b367-d456-423c-8733-de45311c1a42\") "
Jul 17 18:15:07.951423 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:07.951396    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8735b367-d456-423c-8733-de45311c1a42-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8735b367-d456-423c-8733-de45311c1a42" (UID: "8735b367-d456-423c-8733-de45311c1a42"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:15:08.052032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:08.051991    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8735b367-d456-423c-8733-de45311c1a42-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:15:08.712586 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:08.712499    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4" event={"ID":"8735b367-d456-423c-8733-de45311c1a42","Type":"ContainerDied","Data":"f03000bf34958dad23fe5aed7535fbb660ad8604216ce6a7a2175a86c44bef18"}
Jul 17 18:15:08.712586 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:08.712565    2572 scope.go:117] "RemoveContainer" containerID="3be7d8c967efd81f3bd922a83140bfd9590197f92d8fd6fb51a6e15d5c991dba"
Jul 17 18:15:08.712827 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:08.712513    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"
Jul 17 18:15:08.714628 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:08.714600    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472" event={"ID":"49b8ba94-5cc8-4a8d-b7a4-a72aad654786","Type":"ContainerStarted","Data":"6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d"}
Jul 17 18:15:08.714771 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:08.714640    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472" event={"ID":"49b8ba94-5cc8-4a8d-b7a4-a72aad654786","Type":"ContainerStarted","Data":"7a7e386029646b9a84c680c2cd609da8e3dc83909f2db7bbdd6d1f85b60ac01f"}
Jul 17 18:15:08.730306 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:08.730267    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"]
Jul 17 18:15:08.736205 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:08.736173    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-hl7l4"]
Jul 17 18:15:08.754314 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:08.754258    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472" podStartSLOduration=1.754240733 podStartE2EDuration="1.754240733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:15:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:15:08.751867051 +0000 UTC m=+3080.757222925" watchObservedRunningTime="2026-07-17 18:15:08.754240733 +0000 UTC m=+3080.759596587"
Jul 17 18:15:10.515732 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:10.515697    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8735b367-d456-423c-8733-de45311c1a42" path="/var/lib/kubelet/pods/8735b367-d456-423c-8733-de45311c1a42/volumes"
Jul 17 18:15:14.661365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.661322    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"]
Jul 17 18:15:14.661769 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.661670    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8735b367-d456-423c-8733-de45311c1a42" containerName="revoke-keys"
Jul 17 18:15:14.661769 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.661681    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8735b367-d456-423c-8733-de45311c1a42" containerName="revoke-keys"
Jul 17 18:15:14.661769 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.661695    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8735b367-d456-423c-8733-de45311c1a42" containerName="revoke-keys"
Jul 17 18:15:14.661769 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.661701    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8735b367-d456-423c-8733-de45311c1a42" containerName="revoke-keys"
Jul 17 18:15:14.661912 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.661774    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8735b367-d456-423c-8733-de45311c1a42" containerName="revoke-keys"
Jul 17 18:15:14.661912 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.661783    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8735b367-d456-423c-8733-de45311c1a42" containerName="revoke-keys"
Jul 17 18:15:14.666428 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.666406    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"
Jul 17 18:15:14.669614 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.669582    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:15:14.669823 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.669637    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:15:14.672379 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.672349    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"]
Jul 17 18:15:14.709674 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.709629    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8379c599-9d0d-4839-8c4b-9ce350ba0c27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5\" (UID: \"8379c599-9d0d-4839-8c4b-9ce350ba0c27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"
Jul 17 18:15:14.810742 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.810696    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8379c599-9d0d-4839-8c4b-9ce350ba0c27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5\" (UID: \"8379c599-9d0d-4839-8c4b-9ce350ba0c27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"
Jul 17 18:15:14.811397 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.811369    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8379c599-9d0d-4839-8c4b-9ce350ba0c27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5\" (UID: \"8379c599-9d0d-4839-8c4b-9ce350ba0c27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"
Jul 17 18:15:14.981018 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:14.980918    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"
Jul 17 18:15:15.121430 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:15.121398    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"]
Jul 17 18:15:15.123687 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:15:15.123653    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8379c599_9d0d_4839_8c4b_9ce350ba0c27.slice/crio-46f147f16c976e3faa69d904f3b1c05c5be180a3318a000dc36d2def6607bb17 WatchSource:0}: Error finding container 46f147f16c976e3faa69d904f3b1c05c5be180a3318a000dc36d2def6607bb17: Status 404 returned error can't find the container with id 46f147f16c976e3faa69d904f3b1c05c5be180a3318a000dc36d2def6607bb17
Jul 17 18:15:15.756606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:15.756511    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" event={"ID":"8379c599-9d0d-4839-8c4b-9ce350ba0c27","Type":"ContainerStarted","Data":"3c96bac367d11fa7f7942c87988adc2701975d5449be5899f2a2bae0fb0e8f17"}
Jul 17 18:15:15.756606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:15.756566    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" event={"ID":"8379c599-9d0d-4839-8c4b-9ce350ba0c27","Type":"ContainerStarted","Data":"46f147f16c976e3faa69d904f3b1c05c5be180a3318a000dc36d2def6607bb17"}
Jul 17 18:15:15.775681 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:15.775615    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" podStartSLOduration=1.7755961230000001 podStartE2EDuration="1.775596123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:15:15.773789254 +0000 UTC m=+3087.779145108" watchObservedRunningTime="2026-07-17 18:15:15.775596123 +0000 UTC m=+3087.780952037"
Jul 17 18:15:16.764059 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:16.764022    2572 generic.go:356] "Generic (PLEG): container finished" podID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerID="3c96bac367d11fa7f7942c87988adc2701975d5449be5899f2a2bae0fb0e8f17" exitCode=6
Jul 17 18:15:16.764520 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:16.764106    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" event={"ID":"8379c599-9d0d-4839-8c4b-9ce350ba0c27","Type":"ContainerDied","Data":"3c96bac367d11fa7f7942c87988adc2701975d5449be5899f2a2bae0fb0e8f17"}
Jul 17 18:15:16.764520 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:16.764299    2572 scope.go:117] "RemoveContainer" containerID="3c96bac367d11fa7f7942c87988adc2701975d5449be5899f2a2bae0fb0e8f17"
Jul 17 18:15:17.603993 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:17.603878    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:17.603993 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:17.603937    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:17.616755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:17.616722    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:17.771600 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:17.771566    2572 generic.go:356] "Generic (PLEG): container finished" podID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerID="089ab2870706491bd0d895da30edc6cb9b97e695477fb317d7d83bc12160bd62" exitCode=6
Jul 17 18:15:17.772063 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:17.771643    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" event={"ID":"8379c599-9d0d-4839-8c4b-9ce350ba0c27","Type":"ContainerDied","Data":"089ab2870706491bd0d895da30edc6cb9b97e695477fb317d7d83bc12160bd62"}
Jul 17 18:15:17.772063 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:17.771700    2572 scope.go:117] "RemoveContainer" containerID="3c96bac367d11fa7f7942c87988adc2701975d5449be5899f2a2bae0fb0e8f17"
Jul 17 18:15:17.772063 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:17.771912    2572 scope.go:117] "RemoveContainer" containerID="089ab2870706491bd0d895da30edc6cb9b97e695477fb317d7d83bc12160bd62"
Jul 17 18:15:17.772220 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:17.772126    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5_odh-ai-gateway-infra(8379c599-9d0d-4839-8c4b-9ce350ba0c27)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27"
Jul 17 18:15:17.783687 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:17.783656    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:15:18.779079 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:18.779046    2572 scope.go:117] "RemoveContainer" containerID="089ab2870706491bd0d895da30edc6cb9b97e695477fb317d7d83bc12160bd62"
Jul 17 18:15:18.779483 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:18.779297    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5_odh-ai-gateway-infra(8379c599-9d0d-4839-8c4b-9ce350ba0c27)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27"
Jul 17 18:15:20.460402 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.460364    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"]
Jul 17 18:15:20.460806 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.460727    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8735b367-d456-423c-8733-de45311c1a42" containerName="revoke-keys"
Jul 17 18:15:20.460806 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.460739    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8735b367-d456-423c-8733-de45311c1a42" containerName="revoke-keys"
Jul 17 18:15:20.460806 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.460799    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8735b367-d456-423c-8733-de45311c1a42" containerName="revoke-keys"
Jul 17 18:15:20.465032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.465009    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"
Jul 17 18:15:20.472470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.472432    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"]
Jul 17 18:15:20.553792 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.553746    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7e601b6-285b-4f84-92d0-13d803864236-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh\" (UID: \"b7e601b6-285b-4f84-92d0-13d803864236\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"
Jul 17 18:15:20.654460 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.654417    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7e601b6-285b-4f84-92d0-13d803864236-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh\" (UID: \"b7e601b6-285b-4f84-92d0-13d803864236\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"
Jul 17 18:15:20.655076 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.655054    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7e601b6-285b-4f84-92d0-13d803864236-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh\" (UID: \"b7e601b6-285b-4f84-92d0-13d803864236\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"
Jul 17 18:15:20.777437 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:20.777336    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"
Jul 17 18:15:21.128110 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:21.128077    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"]
Jul 17 18:15:21.129589 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:15:21.129545    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e601b6_285b_4f84_92d0_13d803864236.slice/crio-c69f750624497f1cb9ae119fcc5cb26ae9c036d0b12110d1b7123ce27fabfae2 WatchSource:0}: Error finding container c69f750624497f1cb9ae119fcc5cb26ae9c036d0b12110d1b7123ce27fabfae2: Status 404 returned error can't find the container with id c69f750624497f1cb9ae119fcc5cb26ae9c036d0b12110d1b7123ce27fabfae2
Jul 17 18:15:21.798824 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:21.798786    2572 generic.go:356] "Generic (PLEG): container finished" podID="b7e601b6-285b-4f84-92d0-13d803864236" containerID="256c36b92a5cd594bf6f317ebeee8fc897117e912e7f56eed87bff19b4dce736" exitCode=6
Jul 17 18:15:21.799286 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:21.798869    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" event={"ID":"b7e601b6-285b-4f84-92d0-13d803864236","Type":"ContainerDied","Data":"256c36b92a5cd594bf6f317ebeee8fc897117e912e7f56eed87bff19b4dce736"}
Jul 17 18:15:21.799286 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:21.798926    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" event={"ID":"b7e601b6-285b-4f84-92d0-13d803864236","Type":"ContainerStarted","Data":"c69f750624497f1cb9ae119fcc5cb26ae9c036d0b12110d1b7123ce27fabfae2"}
Jul 17 18:15:21.799286 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:21.799032    2572 scope.go:117] "RemoveContainer" containerID="256c36b92a5cd594bf6f317ebeee8fc897117e912e7f56eed87bff19b4dce736"
Jul 17 18:15:22.806054 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:22.806020    2572 generic.go:356] "Generic (PLEG): container finished" podID="b7e601b6-285b-4f84-92d0-13d803864236" containerID="ff8e64686582b484d6f1d595c1e2e0c49e28cd6c0a79d1e727f6aa389bdb217d" exitCode=6
Jul 17 18:15:22.806569 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:22.806100    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" event={"ID":"b7e601b6-285b-4f84-92d0-13d803864236","Type":"ContainerDied","Data":"ff8e64686582b484d6f1d595c1e2e0c49e28cd6c0a79d1e727f6aa389bdb217d"}
Jul 17 18:15:22.806569 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:22.806161    2572 scope.go:117] "RemoveContainer" containerID="256c36b92a5cd594bf6f317ebeee8fc897117e912e7f56eed87bff19b4dce736"
Jul 17 18:15:22.806569 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:22.806388    2572 scope.go:117] "RemoveContainer" containerID="ff8e64686582b484d6f1d595c1e2e0c49e28cd6c0a79d1e727f6aa389bdb217d"
Jul 17 18:15:22.806717 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:22.806683    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh_odh-ai-gateway-infra(b7e601b6-285b-4f84-92d0-13d803864236)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" podUID="b7e601b6-285b-4f84-92d0-13d803864236"
Jul 17 18:15:23.813985 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:23.813954    2572 scope.go:117] "RemoveContainer" containerID="ff8e64686582b484d6f1d595c1e2e0c49e28cd6c0a79d1e727f6aa389bdb217d"
Jul 17 18:15:23.814391 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:23.814138    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh_odh-ai-gateway-infra(b7e601b6-285b-4f84-92d0-13d803864236)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" podUID="b7e601b6-285b-4f84-92d0-13d803864236"
Jul 17 18:15:28.820644 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:28.820603    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"]
Jul 17 18:15:28.825502 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:28.825474    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"
Jul 17 18:15:28.832218 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:28.832185    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"]
Jul 17 18:15:28.928732 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:28.928686    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6887b18-ebf7-4b11-9bd6-60172c443693-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b\" (UID: \"b6887b18-ebf7-4b11-9bd6-60172c443693\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"
Jul 17 18:15:29.029770 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:29.029725    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6887b18-ebf7-4b11-9bd6-60172c443693-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b\" (UID: \"b6887b18-ebf7-4b11-9bd6-60172c443693\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"
Jul 17 18:15:29.030439 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:29.030407    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6887b18-ebf7-4b11-9bd6-60172c443693-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b\" (UID: \"b6887b18-ebf7-4b11-9bd6-60172c443693\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"
Jul 17 18:15:29.137260 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:29.137165    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"
Jul 17 18:15:29.278849 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:29.278821    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"]
Jul 17 18:15:29.280152 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:15:29.280121    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6887b18_ebf7_4b11_9bd6_60172c443693.slice/crio-0929ab55aef50d3d5ee3bb3306ffc8e0eb9848303329162d11553fc353cee9cd WatchSource:0}: Error finding container 0929ab55aef50d3d5ee3bb3306ffc8e0eb9848303329162d11553fc353cee9cd: Status 404 returned error can't find the container with id 0929ab55aef50d3d5ee3bb3306ffc8e0eb9848303329162d11553fc353cee9cd
Jul 17 18:15:29.857629 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:29.857593    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerID="f68fbdfea8ce103e23833d040ed84cc0bb849d3076aa288350d8a8197f8f5ba0" exitCode=6
Jul 17 18:15:29.858122 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:29.857670    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" event={"ID":"b6887b18-ebf7-4b11-9bd6-60172c443693","Type":"ContainerDied","Data":"f68fbdfea8ce103e23833d040ed84cc0bb849d3076aa288350d8a8197f8f5ba0"}
Jul 17 18:15:29.858122 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:29.857724    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" event={"ID":"b6887b18-ebf7-4b11-9bd6-60172c443693","Type":"ContainerStarted","Data":"0929ab55aef50d3d5ee3bb3306ffc8e0eb9848303329162d11553fc353cee9cd"}
Jul 17 18:15:29.858122 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:29.857950    2572 scope.go:117] "RemoveContainer" containerID="f68fbdfea8ce103e23833d040ed84cc0bb849d3076aa288350d8a8197f8f5ba0"
Jul 17 18:15:30.864969 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:30.864932    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerID="0df30b0d39863eab3dde97cd95f747bea854307a7a1915ec20f859420350ff56" exitCode=6
Jul 17 18:15:30.865468 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:30.865007    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" event={"ID":"b6887b18-ebf7-4b11-9bd6-60172c443693","Type":"ContainerDied","Data":"0df30b0d39863eab3dde97cd95f747bea854307a7a1915ec20f859420350ff56"}
Jul 17 18:15:30.865468 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:30.865055    2572 scope.go:117] "RemoveContainer" containerID="f68fbdfea8ce103e23833d040ed84cc0bb849d3076aa288350d8a8197f8f5ba0"
Jul 17 18:15:30.865468 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:30.865228    2572 scope.go:117] "RemoveContainer" containerID="0df30b0d39863eab3dde97cd95f747bea854307a7a1915ec20f859420350ff56"
Jul 17 18:15:30.865677 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:30.865523    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b_odh-ai-gateway-infra(b6887b18-ebf7-4b11-9bd6-60172c443693)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693"
Jul 17 18:15:31.872899 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:31.872866    2572 scope.go:117] "RemoveContainer" containerID="0df30b0d39863eab3dde97cd95f747bea854307a7a1915ec20f859420350ff56"
Jul 17 18:15:31.873304 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:31.873062    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b_odh-ai-gateway-infra(b6887b18-ebf7-4b11-9bd6-60172c443693)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693"
Jul 17 18:15:33.509893 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:33.509859    2572 scope.go:117] "RemoveContainer" containerID="089ab2870706491bd0d895da30edc6cb9b97e695477fb317d7d83bc12160bd62"
Jul 17 18:15:33.886229 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:33.886194    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" event={"ID":"8379c599-9d0d-4839-8c4b-9ce350ba0c27","Type":"ContainerStarted","Data":"637dbcad7133f0733e122eb0849864ed5c5bb1796ca15fc9fb2dffef1c4daf14"}
Jul 17 18:15:34.894427 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:34.894385    2572 generic.go:356] "Generic (PLEG): container finished" podID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerID="637dbcad7133f0733e122eb0849864ed5c5bb1796ca15fc9fb2dffef1c4daf14" exitCode=6
Jul 17 18:15:34.894913 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:34.894466    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" event={"ID":"8379c599-9d0d-4839-8c4b-9ce350ba0c27","Type":"ContainerDied","Data":"637dbcad7133f0733e122eb0849864ed5c5bb1796ca15fc9fb2dffef1c4daf14"}
Jul 17 18:15:34.894913 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:34.894548    2572 scope.go:117] "RemoveContainer" containerID="089ab2870706491bd0d895da30edc6cb9b97e695477fb317d7d83bc12160bd62"
Jul 17 18:15:34.894913 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:34.894865    2572 scope.go:117] "RemoveContainer" containerID="637dbcad7133f0733e122eb0849864ed5c5bb1796ca15fc9fb2dffef1c4daf14"
Jul 17 18:15:34.895107 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:34.895088    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5_odh-ai-gateway-infra(8379c599-9d0d-4839-8c4b-9ce350ba0c27)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27"
Jul 17 18:15:34.920791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:34.920749    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"]
Jul 17 18:15:36.043784 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:36.043750    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"
Jul 17 18:15:36.191266 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:36.191157    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8379c599-9d0d-4839-8c4b-9ce350ba0c27-maas-api-service-ca\") pod \"8379c599-9d0d-4839-8c4b-9ce350ba0c27\" (UID: \"8379c599-9d0d-4839-8c4b-9ce350ba0c27\") "
Jul 17 18:15:36.191590 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:36.191567    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8379c599-9d0d-4839-8c4b-9ce350ba0c27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8379c599-9d0d-4839-8c4b-9ce350ba0c27" (UID: "8379c599-9d0d-4839-8c4b-9ce350ba0c27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:15:36.292659 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:36.292617    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8379c599-9d0d-4839-8c4b-9ce350ba0c27-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:15:36.910049 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:36.910013    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"
Jul 17 18:15:36.910049 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:36.910037    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5" event={"ID":"8379c599-9d0d-4839-8c4b-9ce350ba0c27","Type":"ContainerDied","Data":"46f147f16c976e3faa69d904f3b1c05c5be180a3318a000dc36d2def6607bb17"}
Jul 17 18:15:36.910323 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:36.910100    2572 scope.go:117] "RemoveContainer" containerID="637dbcad7133f0733e122eb0849864ed5c5bb1796ca15fc9fb2dffef1c4daf14"
Jul 17 18:15:36.928315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:36.928277    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"]
Jul 17 18:15:36.935684 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:36.935651    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-lfhl5"]
Jul 17 18:15:38.513443 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:38.513415    2572 scope.go:117] "RemoveContainer" containerID="ff8e64686582b484d6f1d595c1e2e0c49e28cd6c0a79d1e727f6aa389bdb217d"
Jul 17 18:15:38.516230 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:38.516205    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" path="/var/lib/kubelet/pods/8379c599-9d0d-4839-8c4b-9ce350ba0c27/volumes"
Jul 17 18:15:38.924802 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:38.924767    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" event={"ID":"b7e601b6-285b-4f84-92d0-13d803864236","Type":"ContainerStarted","Data":"d51aea3065baab353e4523493de725d83d5d92a052a0260fe253bf41898e47ff"}
Jul 17 18:15:38.947571 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:38.947512    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" podStartSLOduration=18.947476944 podStartE2EDuration="18.947476944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:15:38.945039532 +0000 UTC m=+3110.950395389" watchObservedRunningTime="2026-07-17 18:15:38.947476944 +0000 UTC m=+3110.952832797"
Jul 17 18:15:39.932624 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:39.932587    2572 generic.go:356] "Generic (PLEG): container finished" podID="b7e601b6-285b-4f84-92d0-13d803864236" containerID="d51aea3065baab353e4523493de725d83d5d92a052a0260fe253bf41898e47ff" exitCode=6
Jul 17 18:15:39.933032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:39.932658    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" event={"ID":"b7e601b6-285b-4f84-92d0-13d803864236","Type":"ContainerDied","Data":"d51aea3065baab353e4523493de725d83d5d92a052a0260fe253bf41898e47ff"}
Jul 17 18:15:39.933032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:39.932728    2572 scope.go:117] "RemoveContainer" containerID="ff8e64686582b484d6f1d595c1e2e0c49e28cd6c0a79d1e727f6aa389bdb217d"
Jul 17 18:15:39.933032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:39.933014    2572 scope.go:117] "RemoveContainer" containerID="d51aea3065baab353e4523493de725d83d5d92a052a0260fe253bf41898e47ff"
Jul 17 18:15:39.933234 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:39.933209    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh_odh-ai-gateway-infra(b7e601b6-285b-4f84-92d0-13d803864236)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" podUID="b7e601b6-285b-4f84-92d0-13d803864236"
Jul 17 18:15:39.961988 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:39.961950    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"]
Jul 17 18:15:41.099952 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:41.099923    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"
Jul 17 18:15:41.240174 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:41.240074    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7e601b6-285b-4f84-92d0-13d803864236-maas-api-service-ca\") pod \"b7e601b6-285b-4f84-92d0-13d803864236\" (UID: \"b7e601b6-285b-4f84-92d0-13d803864236\") "
Jul 17 18:15:41.240532 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:41.240477    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7e601b6-285b-4f84-92d0-13d803864236-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7e601b6-285b-4f84-92d0-13d803864236" (UID: "b7e601b6-285b-4f84-92d0-13d803864236"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:15:41.341141 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:41.341094    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7e601b6-285b-4f84-92d0-13d803864236-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:15:41.947511 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:41.947455    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"
Jul 17 18:15:41.947708 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:41.947513    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh" event={"ID":"b7e601b6-285b-4f84-92d0-13d803864236","Type":"ContainerDied","Data":"c69f750624497f1cb9ae119fcc5cb26ae9c036d0b12110d1b7123ce27fabfae2"}
Jul 17 18:15:41.947708 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:41.947582    2572 scope.go:117] "RemoveContainer" containerID="d51aea3065baab353e4523493de725d83d5d92a052a0260fe253bf41898e47ff"
Jul 17 18:15:41.972792 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:41.972737    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"]
Jul 17 18:15:41.979137 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:41.979103    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-vsrfh"]
Jul 17 18:15:42.515911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:42.515874    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7e601b6-285b-4f84-92d0-13d803864236" path="/var/lib/kubelet/pods/b7e601b6-285b-4f84-92d0-13d803864236/volumes"
Jul 17 18:15:45.510471 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:45.510430    2572 scope.go:117] "RemoveContainer" containerID="0df30b0d39863eab3dde97cd95f747bea854307a7a1915ec20f859420350ff56"
Jul 17 18:15:45.973533 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:45.973479    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" event={"ID":"b6887b18-ebf7-4b11-9bd6-60172c443693","Type":"ContainerStarted","Data":"8e308b023eaec912a33fd3adbd132735db48d9f739c6eff06438bcf7e413e69a"}
Jul 17 18:15:45.992238 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:45.992180    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" podStartSLOduration=17.992163442 podStartE2EDuration="17.992163442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:15:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:15:45.990677854 +0000 UTC m=+3117.996033721" watchObservedRunningTime="2026-07-17 18:15:45.992163442 +0000 UTC m=+3117.997519296"
Jul 17 18:15:46.928996 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.928948    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"]
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929318    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e601b6-285b-4f84-92d0-13d803864236" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929330    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e601b6-285b-4f84-92d0-13d803864236" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929341    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e601b6-285b-4f84-92d0-13d803864236" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929347    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e601b6-285b-4f84-92d0-13d803864236" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929354    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929359    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929366    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e601b6-285b-4f84-92d0-13d803864236" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929371    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e601b6-285b-4f84-92d0-13d803864236" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929379    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929384    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929396    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929401    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929458    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e601b6-285b-4f84-92d0-13d803864236" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929468    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929475    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929480    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8379c599-9d0d-4839-8c4b-9ce350ba0c27" containerName="revoke-keys"
Jul 17 18:15:46.929482 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.929502    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e601b6-285b-4f84-92d0-13d803864236" containerName="revoke-keys"
Jul 17 18:15:46.932515 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.932485    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"
Jul 17 18:15:46.941750 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.941718    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"]
Jul 17 18:15:46.980813 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.980775    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerID="8e308b023eaec912a33fd3adbd132735db48d9f739c6eff06438bcf7e413e69a" exitCode=6
Jul 17 18:15:46.981017 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.980845    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" event={"ID":"b6887b18-ebf7-4b11-9bd6-60172c443693","Type":"ContainerDied","Data":"8e308b023eaec912a33fd3adbd132735db48d9f739c6eff06438bcf7e413e69a"}
Jul 17 18:15:46.981017 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.980907    2572 scope.go:117] "RemoveContainer" containerID="0df30b0d39863eab3dde97cd95f747bea854307a7a1915ec20f859420350ff56"
Jul 17 18:15:46.981268 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:46.981247    2572 scope.go:117] "RemoveContainer" containerID="8e308b023eaec912a33fd3adbd132735db48d9f739c6eff06438bcf7e413e69a"
Jul 17 18:15:46.981526 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:46.981475    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b_odh-ai-gateway-infra(b6887b18-ebf7-4b11-9bd6-60172c443693)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693"
Jul 17 18:15:47.007961 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:47.007919    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"]
Jul 17 18:15:47.094770 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:47.094728    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/741d721a-1058-4b7f-ab8d-dcf1e6573775-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr\" (UID: \"741d721a-1058-4b7f-ab8d-dcf1e6573775\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"
Jul 17 18:15:47.195983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:47.195886    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/741d721a-1058-4b7f-ab8d-dcf1e6573775-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr\" (UID: \"741d721a-1058-4b7f-ab8d-dcf1e6573775\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"
Jul 17 18:15:47.196462 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:47.196441    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/741d721a-1058-4b7f-ab8d-dcf1e6573775-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr\" (UID: \"741d721a-1058-4b7f-ab8d-dcf1e6573775\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"
Jul 17 18:15:47.243945 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:47.243905    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"
Jul 17 18:15:47.385652 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:47.385620    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"]
Jul 17 18:15:47.387410 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:15:47.387378    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741d721a_1058_4b7f_ab8d_dcf1e6573775.slice/crio-89195f1cd905816946fcc03a686efdfc74889355b4828c6807d47d77c770d390 WatchSource:0}: Error finding container 89195f1cd905816946fcc03a686efdfc74889355b4828c6807d47d77c770d390: Status 404 returned error can't find the container with id 89195f1cd905816946fcc03a686efdfc74889355b4828c6807d47d77c770d390
Jul 17 18:15:47.990426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:47.990387    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" event={"ID":"741d721a-1058-4b7f-ab8d-dcf1e6573775","Type":"ContainerStarted","Data":"c55ae6ffb431ba9226101ca235d53da1226d9a1e333eb3c69c65039c2844842f"}
Jul 17 18:15:47.990789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:47.990462    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" event={"ID":"741d721a-1058-4b7f-ab8d-dcf1e6573775","Type":"ContainerStarted","Data":"89195f1cd905816946fcc03a686efdfc74889355b4828c6807d47d77c770d390"}
Jul 17 18:15:48.023245 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.023183    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" podStartSLOduration=2.023163667 podStartE2EDuration="2.023163667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:15:48.021203733 +0000 UTC m=+3120.026559588" watchObservedRunningTime="2026-07-17 18:15:48.023163667 +0000 UTC m=+3120.028519521"
Jul 17 18:15:48.125150 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.125119    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"
Jul 17 18:15:48.305101 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.305059    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6887b18-ebf7-4b11-9bd6-60172c443693-maas-api-service-ca\") pod \"b6887b18-ebf7-4b11-9bd6-60172c443693\" (UID: \"b6887b18-ebf7-4b11-9bd6-60172c443693\") "
Jul 17 18:15:48.305426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.305404    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6887b18-ebf7-4b11-9bd6-60172c443693-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b6887b18-ebf7-4b11-9bd6-60172c443693" (UID: "b6887b18-ebf7-4b11-9bd6-60172c443693"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:15:48.406307 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.406270    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6887b18-ebf7-4b11-9bd6-60172c443693-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:15:48.998052 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.998015    2572 generic.go:356] "Generic (PLEG): container finished" podID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerID="c55ae6ffb431ba9226101ca235d53da1226d9a1e333eb3c69c65039c2844842f" exitCode=6
Jul 17 18:15:48.998567 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.998109    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" event={"ID":"741d721a-1058-4b7f-ab8d-dcf1e6573775","Type":"ContainerDied","Data":"c55ae6ffb431ba9226101ca235d53da1226d9a1e333eb3c69c65039c2844842f"}
Jul 17 18:15:48.998567 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.998302    2572 scope.go:117] "RemoveContainer" containerID="c55ae6ffb431ba9226101ca235d53da1226d9a1e333eb3c69c65039c2844842f"
Jul 17 18:15:48.999635 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.999616    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b" event={"ID":"b6887b18-ebf7-4b11-9bd6-60172c443693","Type":"ContainerDied","Data":"0929ab55aef50d3d5ee3bb3306ffc8e0eb9848303329162d11553fc353cee9cd"}
Jul 17 18:15:48.999731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.999661    2572 scope.go:117] "RemoveContainer" containerID="8e308b023eaec912a33fd3adbd132735db48d9f739c6eff06438bcf7e413e69a"
Jul 17 18:15:48.999731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:48.999621    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"
Jul 17 18:15:49.035785 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:49.034001    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"]
Jul 17 18:15:49.038777 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:49.038748    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-hck9b"]
Jul 17 18:15:50.007460 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:50.007357    2572 generic.go:356] "Generic (PLEG): container finished" podID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerID="f86c8573cdadfa42b1d1571ff7cf62405a498dba3fed14ad855f7bd255a62fb4" exitCode=6
Jul 17 18:15:50.007460 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:50.007439    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" event={"ID":"741d721a-1058-4b7f-ab8d-dcf1e6573775","Type":"ContainerDied","Data":"f86c8573cdadfa42b1d1571ff7cf62405a498dba3fed14ad855f7bd255a62fb4"}
Jul 17 18:15:50.008001 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:50.007520    2572 scope.go:117] "RemoveContainer" containerID="c55ae6ffb431ba9226101ca235d53da1226d9a1e333eb3c69c65039c2844842f"
Jul 17 18:15:50.008001 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:50.007701    2572 scope.go:117] "RemoveContainer" containerID="f86c8573cdadfa42b1d1571ff7cf62405a498dba3fed14ad855f7bd255a62fb4"
Jul 17 18:15:50.008001 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:50.007965    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr_odh-ai-gateway-infra(741d721a-1058-4b7f-ab8d-dcf1e6573775)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775"
Jul 17 18:15:50.515938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:50.515901    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" path="/var/lib/kubelet/pods/b6887b18-ebf7-4b11-9bd6-60172c443693/volumes"
Jul 17 18:15:51.014371 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:15:51.014347    2572 scope.go:117] "RemoveContainer" containerID="f86c8573cdadfa42b1d1571ff7cf62405a498dba3fed14ad855f7bd255a62fb4"
Jul 17 18:15:51.014791 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:15:51.014589    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr_odh-ai-gateway-infra(741d721a-1058-4b7f-ab8d-dcf1e6573775)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775"
Jul 17 18:16:01.510331 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:01.510297    2572 scope.go:117] "RemoveContainer" containerID="f86c8573cdadfa42b1d1571ff7cf62405a498dba3fed14ad855f7bd255a62fb4"
Jul 17 18:16:02.078911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:02.078873    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" event={"ID":"741d721a-1058-4b7f-ab8d-dcf1e6573775","Type":"ContainerStarted","Data":"806ffb905f216761dcd0159d3b3fae126093652e98aedc7ec3bee9501eff253f"}
Jul 17 18:16:03.086342 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:03.086306    2572 generic.go:356] "Generic (PLEG): container finished" podID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerID="806ffb905f216761dcd0159d3b3fae126093652e98aedc7ec3bee9501eff253f" exitCode=6
Jul 17 18:16:03.086812 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:03.086379    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" event={"ID":"741d721a-1058-4b7f-ab8d-dcf1e6573775","Type":"ContainerDied","Data":"806ffb905f216761dcd0159d3b3fae126093652e98aedc7ec3bee9501eff253f"}
Jul 17 18:16:03.086812 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:03.086459    2572 scope.go:117] "RemoveContainer" containerID="f86c8573cdadfa42b1d1571ff7cf62405a498dba3fed14ad855f7bd255a62fb4"
Jul 17 18:16:03.086812 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:03.086787    2572 scope.go:117] "RemoveContainer" containerID="806ffb905f216761dcd0159d3b3fae126093652e98aedc7ec3bee9501eff253f"
Jul 17 18:16:03.087038 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:03.087017    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr_odh-ai-gateway-infra(741d721a-1058-4b7f-ab8d-dcf1e6573775)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775"
Jul 17 18:16:03.107214 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:03.107173    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"]
Jul 17 18:16:04.233678 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:04.233649    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"
Jul 17 18:16:04.349014 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:04.348908    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/741d721a-1058-4b7f-ab8d-dcf1e6573775-maas-api-service-ca\") pod \"741d721a-1058-4b7f-ab8d-dcf1e6573775\" (UID: \"741d721a-1058-4b7f-ab8d-dcf1e6573775\") "
Jul 17 18:16:04.349319 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:04.349291    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/741d721a-1058-4b7f-ab8d-dcf1e6573775-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "741d721a-1058-4b7f-ab8d-dcf1e6573775" (UID: "741d721a-1058-4b7f-ab8d-dcf1e6573775"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:16:04.449980 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:04.449937    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/741d721a-1058-4b7f-ab8d-dcf1e6573775-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:16:05.100125 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:05.100093    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"
Jul 17 18:16:05.100339 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:05.100092    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr" event={"ID":"741d721a-1058-4b7f-ab8d-dcf1e6573775","Type":"ContainerDied","Data":"89195f1cd905816946fcc03a686efdfc74889355b4828c6807d47d77c770d390"}
Jul 17 18:16:05.100339 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:05.100244    2572 scope.go:117] "RemoveContainer" containerID="806ffb905f216761dcd0159d3b3fae126093652e98aedc7ec3bee9501eff253f"
Jul 17 18:16:05.116990 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:05.116958    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"]
Jul 17 18:16:05.126211 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:05.126175    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-qjcdr"]
Jul 17 18:16:06.516556 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:06.516485    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" path="/var/lib/kubelet/pods/741d721a-1058-4b7f-ab8d-dcf1e6573775/volumes"
Jul 17 18:16:07.383504 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.383453    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6svk9"]
Jul 17 18:16:07.383854 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.383836    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerName="revoke-keys"
Jul 17 18:16:07.383895 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.383859    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerName="revoke-keys"
Jul 17 18:16:07.383939 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.383893    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerName="revoke-keys"
Jul 17 18:16:07.383939 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.383903    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerName="revoke-keys"
Jul 17 18:16:07.383939 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.383921    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerName="revoke-keys"
Jul 17 18:16:07.383939 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.383930    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerName="revoke-keys"
Jul 17 18:16:07.384061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.383949    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerName="revoke-keys"
Jul 17 18:16:07.384061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.383957    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerName="revoke-keys"
Jul 17 18:16:07.384061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.384039    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerName="revoke-keys"
Jul 17 18:16:07.384061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.384050    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerName="revoke-keys"
Jul 17 18:16:07.384061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.384058    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerName="revoke-keys"
Jul 17 18:16:07.384205 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.384070    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerName="revoke-keys"
Jul 17 18:16:07.384205 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.384078    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerName="revoke-keys"
Jul 17 18:16:07.384205 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.384087    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e601b6-285b-4f84-92d0-13d803864236" containerName="revoke-keys"
Jul 17 18:16:07.388731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.388694    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:07.392826 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.392794    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-r7wrj\""
Jul 17 18:16:07.395633 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.395607    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6svk9"]
Jul 17 18:16:07.477477 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.477435    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4phcd\" (UniqueName: \"kubernetes.io/projected/cc25a599-1ba4-49cd-9842-ded2c3bae565-kube-api-access-4phcd\") pod \"kuadrant-operator-catalog-6svk9\" (UID: \"cc25a599-1ba4-49cd-9842-ded2c3bae565\") " pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:07.578140 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.578088    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4phcd\" (UniqueName: \"kubernetes.io/projected/cc25a599-1ba4-49cd-9842-ded2c3bae565-kube-api-access-4phcd\") pod \"kuadrant-operator-catalog-6svk9\" (UID: \"cc25a599-1ba4-49cd-9842-ded2c3bae565\") " pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:07.587739 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.587709    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4phcd\" (UniqueName: \"kubernetes.io/projected/cc25a599-1ba4-49cd-9842-ded2c3bae565-kube-api-access-4phcd\") pod \"kuadrant-operator-catalog-6svk9\" (UID: \"cc25a599-1ba4-49cd-9842-ded2c3bae565\") " pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:07.700037 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.699937    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:07.850291 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.850149    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6svk9"]
Jul 17 18:16:07.853132 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:16:07.853099    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc25a599_1ba4_49cd_9842_ded2c3bae565.slice/crio-d6fd4387b365fa6b318ddfca80f7d68ade3498faac9e37e4726a54b1e6438318 WatchSource:0}: Error finding container d6fd4387b365fa6b318ddfca80f7d68ade3498faac9e37e4726a54b1e6438318: Status 404 returned error can't find the container with id d6fd4387b365fa6b318ddfca80f7d68ade3498faac9e37e4726a54b1e6438318
Jul 17 18:16:07.854573 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:07.854555    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 18:16:08.125056 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:08.125018    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6svk9" event={"ID":"cc25a599-1ba4-49cd-9842-ded2c3bae565","Type":"ContainerStarted","Data":"d6fd4387b365fa6b318ddfca80f7d68ade3498faac9e37e4726a54b1e6438318"}
Jul 17 18:16:10.142237 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:10.142196    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6svk9" event={"ID":"cc25a599-1ba4-49cd-9842-ded2c3bae565","Type":"ContainerStarted","Data":"a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3"}
Jul 17 18:16:10.159325 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:10.159269    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6svk9" podStartSLOduration=1.566519681 podStartE2EDuration="3.1592512s" totalImagesPullingTime="1.592731519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:16:10.158197156 +0000 UTC m=+3142.163553011" watchObservedRunningTime="2026-07-17 18:16:10.1592512 +0000 UTC m=+3142.164607053"
Jul 17 18:16:14.751421 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.751381    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"]
Jul 17 18:16:14.751916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.751748    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerName="revoke-keys"
Jul 17 18:16:14.751916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.751760    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerName="revoke-keys"
Jul 17 18:16:14.751916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.751769    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerName="revoke-keys"
Jul 17 18:16:14.751916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.751774    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6887b18-ebf7-4b11-9bd6-60172c443693" containerName="revoke-keys"
Jul 17 18:16:14.751916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.751844    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="741d721a-1058-4b7f-ab8d-dcf1e6573775" containerName="revoke-keys"
Jul 17 18:16:14.755113 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.755094    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"
Jul 17 18:16:14.758045 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.758015    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:16:14.758276 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.758262    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:16:14.765069 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.765039    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"]
Jul 17 18:16:14.852621 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.852580    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd79e45-a98b-4419-b3c1-3ec1068d05fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7\" (UID: \"4fd79e45-a98b-4419-b3c1-3ec1068d05fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"
Jul 17 18:16:14.953119 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.953080    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd79e45-a98b-4419-b3c1-3ec1068d05fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7\" (UID: \"4fd79e45-a98b-4419-b3c1-3ec1068d05fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"
Jul 17 18:16:14.953789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:14.953766    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd79e45-a98b-4419-b3c1-3ec1068d05fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7\" (UID: \"4fd79e45-a98b-4419-b3c1-3ec1068d05fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"
Jul 17 18:16:15.067217 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:15.067178    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"
Jul 17 18:16:15.208692 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:15.208658    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"]
Jul 17 18:16:15.210641 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:16:15.210604    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd79e45_a98b_4419_b3c1_3ec1068d05fb.slice/crio-0c142554e6bfbb08642efe671f574cbd9506de3d396bd1b77e05eb3c752b91a8 WatchSource:0}: Error finding container 0c142554e6bfbb08642efe671f574cbd9506de3d396bd1b77e05eb3c752b91a8: Status 404 returned error can't find the container with id 0c142554e6bfbb08642efe671f574cbd9506de3d396bd1b77e05eb3c752b91a8
Jul 17 18:16:16.179967 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:16.179879    2572 generic.go:356] "Generic (PLEG): container finished" podID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerID="1145aa36a8bab01d702dad0130252d8a8d8488b1efee92e23e98c34c02c5e3bf" exitCode=6
Jul 17 18:16:16.179967 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:16.179916    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7" event={"ID":"4fd79e45-a98b-4419-b3c1-3ec1068d05fb","Type":"ContainerDied","Data":"1145aa36a8bab01d702dad0130252d8a8d8488b1efee92e23e98c34c02c5e3bf"}
Jul 17 18:16:16.179967 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:16.179970    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7" event={"ID":"4fd79e45-a98b-4419-b3c1-3ec1068d05fb","Type":"ContainerStarted","Data":"0c142554e6bfbb08642efe671f574cbd9506de3d396bd1b77e05eb3c752b91a8"}
Jul 17 18:16:16.180419 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:16.180167    2572 scope.go:117] "RemoveContainer" containerID="1145aa36a8bab01d702dad0130252d8a8d8488b1efee92e23e98c34c02c5e3bf"
Jul 17 18:16:17.187795 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:17.187760    2572 generic.go:356] "Generic (PLEG): container finished" podID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerID="7f445965a27f51ba3aabfe75416af37de47a6179b29362d2213fa24015cf2bd6" exitCode=6
Jul 17 18:16:17.188385 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:17.187847    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7" event={"ID":"4fd79e45-a98b-4419-b3c1-3ec1068d05fb","Type":"ContainerDied","Data":"7f445965a27f51ba3aabfe75416af37de47a6179b29362d2213fa24015cf2bd6"}
Jul 17 18:16:17.188385 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:17.187910    2572 scope.go:117] "RemoveContainer" containerID="1145aa36a8bab01d702dad0130252d8a8d8488b1efee92e23e98c34c02c5e3bf"
Jul 17 18:16:17.188385 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:17.188154    2572 scope.go:117] "RemoveContainer" containerID="7f445965a27f51ba3aabfe75416af37de47a6179b29362d2213fa24015cf2bd6"
Jul 17 18:16:17.188385 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:17.188357    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7_odh-ai-gateway-infra(4fd79e45-a98b-4419-b3c1-3ec1068d05fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb"
Jul 17 18:16:17.701074 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:17.701038    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:17.701288 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:17.701104    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:17.724268 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:17.724238    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:18.195990 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:18.195957    2572 scope.go:117] "RemoveContainer" containerID="7f445965a27f51ba3aabfe75416af37de47a6179b29362d2213fa24015cf2bd6"
Jul 17 18:16:18.196440 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:18.196202    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7_odh-ai-gateway-infra(4fd79e45-a98b-4419-b3c1-3ec1068d05fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb"
Jul 17 18:16:18.219985 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:18.219952    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:18.260228 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:18.260184    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6svk9"]
Jul 17 18:16:20.206214 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.206160    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-6svk9" podUID="cc25a599-1ba4-49cd-9842-ded2c3bae565" containerName="registry-server" containerID="cri-o://a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3" gracePeriod=2
Jul 17 18:16:20.472610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.472582    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:20.507804 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.507767    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"]
Jul 17 18:16:20.508140 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.508126    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc25a599-1ba4-49cd-9842-ded2c3bae565" containerName="registry-server"
Jul 17 18:16:20.508140 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.508140    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc25a599-1ba4-49cd-9842-ded2c3bae565" containerName="registry-server"
Jul 17 18:16:20.508286 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.508200    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc25a599-1ba4-49cd-9842-ded2c3bae565" containerName="registry-server"
Jul 17 18:16:20.511523 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.511480    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"
Jul 17 18:16:20.523289 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.522790    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"]
Jul 17 18:16:20.608314 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.608277    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4phcd\" (UniqueName: \"kubernetes.io/projected/cc25a599-1ba4-49cd-9842-ded2c3bae565-kube-api-access-4phcd\") pod \"cc25a599-1ba4-49cd-9842-ded2c3bae565\" (UID: \"cc25a599-1ba4-49cd-9842-ded2c3bae565\") "
Jul 17 18:16:20.608726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.608700    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae69940f-ba1d-4017-962c-1633a020553a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk\" (UID: \"ae69940f-ba1d-4017-962c-1633a020553a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"
Jul 17 18:16:20.610547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.610524    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc25a599-1ba4-49cd-9842-ded2c3bae565-kube-api-access-4phcd" (OuterVolumeSpecName: "kube-api-access-4phcd") pod "cc25a599-1ba4-49cd-9842-ded2c3bae565" (UID: "cc25a599-1ba4-49cd-9842-ded2c3bae565"). InnerVolumeSpecName "kube-api-access-4phcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:16:20.709965 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.709860    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae69940f-ba1d-4017-962c-1633a020553a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk\" (UID: \"ae69940f-ba1d-4017-962c-1633a020553a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"
Jul 17 18:16:20.710143 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.709976    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4phcd\" (UniqueName: \"kubernetes.io/projected/cc25a599-1ba4-49cd-9842-ded2c3bae565-kube-api-access-4phcd\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:16:20.710566 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.710546    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae69940f-ba1d-4017-962c-1633a020553a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk\" (UID: \"ae69940f-ba1d-4017-962c-1633a020553a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"
Jul 17 18:16:20.825290 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.825243    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"
Jul 17 18:16:20.966952 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:20.966855    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"]
Jul 17 18:16:20.969522 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:16:20.969468    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae69940f_ba1d_4017_962c_1633a020553a.slice/crio-596fc7ec8ae8e99c52143b1bdf7674a44b875451c237ec7b29c426ba248354c5 WatchSource:0}: Error finding container 596fc7ec8ae8e99c52143b1bdf7674a44b875451c237ec7b29c426ba248354c5: Status 404 returned error can't find the container with id 596fc7ec8ae8e99c52143b1bdf7674a44b875451c237ec7b29c426ba248354c5
Jul 17 18:16:21.212681 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.212642    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk" event={"ID":"ae69940f-ba1d-4017-962c-1633a020553a","Type":"ContainerStarted","Data":"596fc7ec8ae8e99c52143b1bdf7674a44b875451c237ec7b29c426ba248354c5"}
Jul 17 18:16:21.214006 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.213979    2572 generic.go:356] "Generic (PLEG): container finished" podID="cc25a599-1ba4-49cd-9842-ded2c3bae565" containerID="a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3" exitCode=0
Jul 17 18:16:21.214104 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.214022    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6svk9"
Jul 17 18:16:21.214104 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.214044    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6svk9" event={"ID":"cc25a599-1ba4-49cd-9842-ded2c3bae565","Type":"ContainerDied","Data":"a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3"}
Jul 17 18:16:21.214104 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.214083    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6svk9" event={"ID":"cc25a599-1ba4-49cd-9842-ded2c3bae565","Type":"ContainerDied","Data":"d6fd4387b365fa6b318ddfca80f7d68ade3498faac9e37e4726a54b1e6438318"}
Jul 17 18:16:21.214257 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.214112    2572 scope.go:117] "RemoveContainer" containerID="a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3"
Jul 17 18:16:21.231673 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.231649    2572 scope.go:117] "RemoveContainer" containerID="a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3"
Jul 17 18:16:21.232011 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:21.231987    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3\": container with ID starting with a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3 not found: ID does not exist" containerID="a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3"
Jul 17 18:16:21.232104 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.232023    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3"} err="failed to get container status \"a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3\": rpc error: code = NotFound desc = could not find container \"a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3\": container with ID starting with a1bc1ec61cec8aaf2c93d73ca05defdc6a936733bb7f53033925e83d205712e3 not found: ID does not exist"
Jul 17 18:16:21.247698 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.247662    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6svk9"]
Jul 17 18:16:21.253927 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:21.253898    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6svk9"]
Jul 17 18:16:22.221722 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:22.221681    2572 generic.go:356] "Generic (PLEG): container finished" podID="ae69940f-ba1d-4017-962c-1633a020553a" containerID="ce312286186129067453a1f8ed2bbe281e6f77ee63db51fd2e41c7081680c2e2" exitCode=6
Jul 17 18:16:22.222223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:22.221760    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk" event={"ID":"ae69940f-ba1d-4017-962c-1633a020553a","Type":"ContainerDied","Data":"ce312286186129067453a1f8ed2bbe281e6f77ee63db51fd2e41c7081680c2e2"}
Jul 17 18:16:22.222223 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:22.222056    2572 scope.go:117] "RemoveContainer" containerID="ce312286186129067453a1f8ed2bbe281e6f77ee63db51fd2e41c7081680c2e2"
Jul 17 18:16:22.516013 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:22.515924    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc25a599-1ba4-49cd-9842-ded2c3bae565" path="/var/lib/kubelet/pods/cc25a599-1ba4-49cd-9842-ded2c3bae565/volumes"
Jul 17 18:16:23.230863 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:23.230766    2572 generic.go:356] "Generic (PLEG): container finished" podID="ae69940f-ba1d-4017-962c-1633a020553a" containerID="63a251a3440b25fc6cbd292830f4978a3ae14eca5d421b9008126c511e3d831c" exitCode=6
Jul 17 18:16:23.230863 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:23.230851    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk" event={"ID":"ae69940f-ba1d-4017-962c-1633a020553a","Type":"ContainerDied","Data":"63a251a3440b25fc6cbd292830f4978a3ae14eca5d421b9008126c511e3d831c"}
Jul 17 18:16:23.231390 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:23.230906    2572 scope.go:117] "RemoveContainer" containerID="ce312286186129067453a1f8ed2bbe281e6f77ee63db51fd2e41c7081680c2e2"
Jul 17 18:16:23.231390 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:23.231090    2572 scope.go:117] "RemoveContainer" containerID="63a251a3440b25fc6cbd292830f4978a3ae14eca5d421b9008126c511e3d831c"
Jul 17 18:16:23.231390 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:23.231359    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk_odh-ai-gateway-infra(ae69940f-ba1d-4017-962c-1633a020553a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk" podUID="ae69940f-ba1d-4017-962c-1633a020553a"
Jul 17 18:16:24.238740 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:24.238713    2572 scope.go:117] "RemoveContainer" containerID="63a251a3440b25fc6cbd292830f4978a3ae14eca5d421b9008126c511e3d831c"
Jul 17 18:16:24.239125 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:24.238894    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk_odh-ai-gateway-infra(ae69940f-ba1d-4017-962c-1633a020553a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk" podUID="ae69940f-ba1d-4017-962c-1633a020553a"
Jul 17 18:16:28.871986 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:28.871934    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"]
Jul 17 18:16:28.877004 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:28.876976    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"
Jul 17 18:16:28.885933 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:28.885905    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"]
Jul 17 18:16:28.983385 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:28.983338    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dc52e6-4802-4fb2-9fe9-ef954ff4c199-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt\" (UID: \"55dc52e6-4802-4fb2-9fe9-ef954ff4c199\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"
Jul 17 18:16:29.084124 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:29.084071    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dc52e6-4802-4fb2-9fe9-ef954ff4c199-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt\" (UID: \"55dc52e6-4802-4fb2-9fe9-ef954ff4c199\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"
Jul 17 18:16:29.084787 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:29.084767    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dc52e6-4802-4fb2-9fe9-ef954ff4c199-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt\" (UID: \"55dc52e6-4802-4fb2-9fe9-ef954ff4c199\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"
Jul 17 18:16:29.188917 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:29.188803    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"
Jul 17 18:16:29.330539 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:29.330508    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"]
Jul 17 18:16:29.332406 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:16:29.332364    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55dc52e6_4802_4fb2_9fe9_ef954ff4c199.slice/crio-d228a4f4609bd3297653e87df77fec78e45e99237341d5f126da631f9dda08bc WatchSource:0}: Error finding container d228a4f4609bd3297653e87df77fec78e45e99237341d5f126da631f9dda08bc: Status 404 returned error can't find the container with id d228a4f4609bd3297653e87df77fec78e45e99237341d5f126da631f9dda08bc
Jul 17 18:16:29.510027 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:29.509990    2572 scope.go:117] "RemoveContainer" containerID="7f445965a27f51ba3aabfe75416af37de47a6179b29362d2213fa24015cf2bd6"
Jul 17 18:16:30.275050 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:30.275012    2572 generic.go:356] "Generic (PLEG): container finished" podID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerID="c4a884d47cf838f1646410e63f52235efec26786f2643a332c60d7c379699799" exitCode=6
Jul 17 18:16:30.275529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:30.275082    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7" event={"ID":"4fd79e45-a98b-4419-b3c1-3ec1068d05fb","Type":"ContainerDied","Data":"c4a884d47cf838f1646410e63f52235efec26786f2643a332c60d7c379699799"}
Jul 17 18:16:30.275529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:30.275152    2572 scope.go:117] "RemoveContainer" containerID="7f445965a27f51ba3aabfe75416af37de47a6179b29362d2213fa24015cf2bd6"
Jul 17 18:16:30.275529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:30.275513    2572 scope.go:117] "RemoveContainer" containerID="c4a884d47cf838f1646410e63f52235efec26786f2643a332c60d7c379699799"
Jul 17 18:16:30.275766 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:30.275743    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7_odh-ai-gateway-infra(4fd79e45-a98b-4419-b3c1-3ec1068d05fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb"
Jul 17 18:16:30.276932 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:30.276909    2572 generic.go:356] "Generic (PLEG): container finished" podID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerID="10185b27fa50bf312a30a71238f1a9152e1be832f5b57d8a767755bc18bda5c5" exitCode=6
Jul 17 18:16:30.277016 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:30.276943    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt" event={"ID":"55dc52e6-4802-4fb2-9fe9-ef954ff4c199","Type":"ContainerDied","Data":"10185b27fa50bf312a30a71238f1a9152e1be832f5b57d8a767755bc18bda5c5"}
Jul 17 18:16:30.277016 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:30.276998    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt" event={"ID":"55dc52e6-4802-4fb2-9fe9-ef954ff4c199","Type":"ContainerStarted","Data":"d228a4f4609bd3297653e87df77fec78e45e99237341d5f126da631f9dda08bc"}
Jul 17 18:16:30.277196 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:30.277183    2572 scope.go:117] "RemoveContainer" containerID="10185b27fa50bf312a30a71238f1a9152e1be832f5b57d8a767755bc18bda5c5"
Jul 17 18:16:31.287048 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:31.286960    2572 generic.go:356] "Generic (PLEG): container finished" podID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerID="c5b5e431986a4dd76e1f5715e73eeed8c53432c8f26ff36532e1ca176bdccdce" exitCode=6
Jul 17 18:16:31.287455 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:31.287044    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt" event={"ID":"55dc52e6-4802-4fb2-9fe9-ef954ff4c199","Type":"ContainerDied","Data":"c5b5e431986a4dd76e1f5715e73eeed8c53432c8f26ff36532e1ca176bdccdce"}
Jul 17 18:16:31.287455 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:31.287100    2572 scope.go:117] "RemoveContainer" containerID="10185b27fa50bf312a30a71238f1a9152e1be832f5b57d8a767755bc18bda5c5"
Jul 17 18:16:31.287455 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:31.287335    2572 scope.go:117] "RemoveContainer" containerID="c5b5e431986a4dd76e1f5715e73eeed8c53432c8f26ff36532e1ca176bdccdce"
Jul 17 18:16:31.287618 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:31.287596    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt_odh-ai-gateway-infra(55dc52e6-4802-4fb2-9fe9-ef954ff4c199)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199"
Jul 17 18:16:31.308584 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:31.308553    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"]
Jul 17 18:16:31.453988 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:31.453958    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"
Jul 17 18:16:31.607601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:31.607560    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd79e45-a98b-4419-b3c1-3ec1068d05fb-maas-api-service-ca\") pod \"4fd79e45-a98b-4419-b3c1-3ec1068d05fb\" (UID: \"4fd79e45-a98b-4419-b3c1-3ec1068d05fb\") "
Jul 17 18:16:31.607951 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:31.607929    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fd79e45-a98b-4419-b3c1-3ec1068d05fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4fd79e45-a98b-4419-b3c1-3ec1068d05fb" (UID: "4fd79e45-a98b-4419-b3c1-3ec1068d05fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:16:31.709129 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:31.709091    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd79e45-a98b-4419-b3c1-3ec1068d05fb-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:16:32.294192 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:32.294162    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"
Jul 17 18:16:32.294678 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:32.294167    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7" event={"ID":"4fd79e45-a98b-4419-b3c1-3ec1068d05fb","Type":"ContainerDied","Data":"0c142554e6bfbb08642efe671f574cbd9506de3d396bd1b77e05eb3c752b91a8"}
Jul 17 18:16:32.294678 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:32.294318    2572 scope.go:117] "RemoveContainer" containerID="c4a884d47cf838f1646410e63f52235efec26786f2643a332c60d7c379699799"
Jul 17 18:16:32.304135 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:32.296725    2572 scope.go:117] "RemoveContainer" containerID="c5b5e431986a4dd76e1f5715e73eeed8c53432c8f26ff36532e1ca176bdccdce"
Jul 17 18:16:32.304135 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:32.297109    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt_odh-ai-gateway-infra(55dc52e6-4802-4fb2-9fe9-ef954ff4c199)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199"
Jul 17 18:16:32.333763 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:32.333721    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"]
Jul 17 18:16:32.336718 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:32.336693    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-j6jd7"]
Jul 17 18:16:32.516046 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:32.516008    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" path="/var/lib/kubelet/pods/4fd79e45-a98b-4419-b3c1-3ec1068d05fb/volumes"
Jul 17 18:16:36.974113 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.974080    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"]
Jul 17 18:16:36.976505 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.974424    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerName="revoke-keys"
Jul 17 18:16:36.976505 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.974436    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerName="revoke-keys"
Jul 17 18:16:36.976505 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.974460    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerName="revoke-keys"
Jul 17 18:16:36.976505 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.974467    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerName="revoke-keys"
Jul 17 18:16:36.976505 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.974530    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerName="revoke-keys"
Jul 17 18:16:36.976505 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.974542    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerName="revoke-keys"
Jul 17 18:16:36.976505 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.974548    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerName="revoke-keys"
Jul 17 18:16:36.977463 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.977448    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"
Jul 17 18:16:36.986960 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:36.986931    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"]
Jul 17 18:16:37.156904 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:37.156862    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92fccef-e0c2-49e7-a5fd-919ea8c404a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-r98kb\" (UID: \"f92fccef-e0c2-49e7-a5fd-919ea8c404a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"
Jul 17 18:16:37.258002 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:37.257911    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92fccef-e0c2-49e7-a5fd-919ea8c404a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-r98kb\" (UID: \"f92fccef-e0c2-49e7-a5fd-919ea8c404a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"
Jul 17 18:16:37.258550 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:37.258531    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92fccef-e0c2-49e7-a5fd-919ea8c404a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-r98kb\" (UID: \"f92fccef-e0c2-49e7-a5fd-919ea8c404a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"
Jul 17 18:16:37.289208 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:37.289162    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"
Jul 17 18:16:37.431475 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:37.431439    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"]
Jul 17 18:16:37.433425 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:16:37.433393    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf92fccef_e0c2_49e7_a5fd_919ea8c404a7.slice/crio-e24f28ff6f542fc828eefb6e9967acb0b294be06396d4e99b6a3a1bb35eb8c39 WatchSource:0}: Error finding container e24f28ff6f542fc828eefb6e9967acb0b294be06396d4e99b6a3a1bb35eb8c39: Status 404 returned error can't find the container with id e24f28ff6f542fc828eefb6e9967acb0b294be06396d4e99b6a3a1bb35eb8c39
Jul 17 18:16:38.333561 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:38.333526    2572 generic.go:356] "Generic (PLEG): container finished" podID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerID="27d54af4aa34665451a9c85f8b11fba27c98b07c9f7efab15a90458432becac9" exitCode=6
Jul 17 18:16:38.334041 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:38.333618    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb" event={"ID":"f92fccef-e0c2-49e7-a5fd-919ea8c404a7","Type":"ContainerDied","Data":"27d54af4aa34665451a9c85f8b11fba27c98b07c9f7efab15a90458432becac9"}
Jul 17 18:16:38.334041 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:38.333675    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb" event={"ID":"f92fccef-e0c2-49e7-a5fd-919ea8c404a7","Type":"ContainerStarted","Data":"e24f28ff6f542fc828eefb6e9967acb0b294be06396d4e99b6a3a1bb35eb8c39"}
Jul 17 18:16:38.334041 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:38.333782    2572 scope.go:117] "RemoveContainer" containerID="27d54af4aa34665451a9c85f8b11fba27c98b07c9f7efab15a90458432becac9"
Jul 17 18:16:39.342883 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:39.342842    2572 generic.go:356] "Generic (PLEG): container finished" podID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerID="34afea826a4917b9514198a6b57becb6776df759eb966db778527739d45cd615" exitCode=6
Jul 17 18:16:39.343339 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:39.342920    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb" event={"ID":"f92fccef-e0c2-49e7-a5fd-919ea8c404a7","Type":"ContainerDied","Data":"34afea826a4917b9514198a6b57becb6776df759eb966db778527739d45cd615"}
Jul 17 18:16:39.343339 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:39.342984    2572 scope.go:117] "RemoveContainer" containerID="27d54af4aa34665451a9c85f8b11fba27c98b07c9f7efab15a90458432becac9"
Jul 17 18:16:39.343339 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:39.343169    2572 scope.go:117] "RemoveContainer" containerID="34afea826a4917b9514198a6b57becb6776df759eb966db778527739d45cd615"
Jul 17 18:16:39.343458 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:39.343434    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-r98kb_odh-ai-gateway-infra(f92fccef-e0c2-49e7-a5fd-919ea8c404a7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7"
Jul 17 18:16:39.510653 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:39.510557    2572 scope.go:117] "RemoveContainer" containerID="63a251a3440b25fc6cbd292830f4978a3ae14eca5d421b9008126c511e3d831c"
Jul 17 18:16:40.350484 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:40.350442    2572 scope.go:117] "RemoveContainer" containerID="34afea826a4917b9514198a6b57becb6776df759eb966db778527739d45cd615"
Jul 17 18:16:40.350935 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:40.350692    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-r98kb_odh-ai-gateway-infra(f92fccef-e0c2-49e7-a5fd-919ea8c404a7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7"
Jul 17 18:16:40.352241 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:40.352213    2572 generic.go:356] "Generic (PLEG): container finished" podID="ae69940f-ba1d-4017-962c-1633a020553a" containerID="193f2cb127c26bfc6fbd88a894a25d05bb5aa74d523530c7801783977d6542ee" exitCode=6
Jul 17 18:16:40.352359 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:40.352281    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk" event={"ID":"ae69940f-ba1d-4017-962c-1633a020553a","Type":"ContainerDied","Data":"193f2cb127c26bfc6fbd88a894a25d05bb5aa74d523530c7801783977d6542ee"}
Jul 17 18:16:40.352359 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:40.352335    2572 scope.go:117] "RemoveContainer" containerID="63a251a3440b25fc6cbd292830f4978a3ae14eca5d421b9008126c511e3d831c"
Jul 17 18:16:40.352592 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:40.352577    2572 scope.go:117] "RemoveContainer" containerID="193f2cb127c26bfc6fbd88a894a25d05bb5aa74d523530c7801783977d6542ee"
Jul 17 18:16:40.352809 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:40.352791    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk_odh-ai-gateway-infra(ae69940f-ba1d-4017-962c-1633a020553a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk" podUID="ae69940f-ba1d-4017-962c-1633a020553a"
Jul 17 18:16:41.399076 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:41.399035    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"]
Jul 17 18:16:41.542747 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:41.542712    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"
Jul 17 18:16:41.594759 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:41.594714    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae69940f-ba1d-4017-962c-1633a020553a-maas-api-service-ca\") pod \"ae69940f-ba1d-4017-962c-1633a020553a\" (UID: \"ae69940f-ba1d-4017-962c-1633a020553a\") "
Jul 17 18:16:41.595060 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:41.595033    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae69940f-ba1d-4017-962c-1633a020553a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae69940f-ba1d-4017-962c-1633a020553a" (UID: "ae69940f-ba1d-4017-962c-1633a020553a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:16:41.695587 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:41.695439    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae69940f-ba1d-4017-962c-1633a020553a-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:16:42.370631 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:42.370589    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk" event={"ID":"ae69940f-ba1d-4017-962c-1633a020553a","Type":"ContainerDied","Data":"596fc7ec8ae8e99c52143b1bdf7674a44b875451c237ec7b29c426ba248354c5"}
Jul 17 18:16:42.370631 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:42.370616    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"
Jul 17 18:16:42.370872 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:42.370704    2572 scope.go:117] "RemoveContainer" containerID="193f2cb127c26bfc6fbd88a894a25d05bb5aa74d523530c7801783977d6542ee"
Jul 17 18:16:42.394108 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:42.394070    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"]
Jul 17 18:16:42.397352 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:42.397313    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-rjvvk"]
Jul 17 18:16:42.515291 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:42.515257    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae69940f-ba1d-4017-962c-1633a020553a" path="/var/lib/kubelet/pods/ae69940f-ba1d-4017-962c-1633a020553a/volumes"
Jul 17 18:16:47.509907 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:47.509871    2572 scope.go:117] "RemoveContainer" containerID="c5b5e431986a4dd76e1f5715e73eeed8c53432c8f26ff36532e1ca176bdccdce"
Jul 17 18:16:48.408523 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:48.408400    2572 generic.go:356] "Generic (PLEG): container finished" podID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerID="5b258586b283b24194daefae3c5ff0b82a10a4d6dc595f1d100d7515572ddbff" exitCode=6
Jul 17 18:16:48.408523 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:48.408473    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt" event={"ID":"55dc52e6-4802-4fb2-9fe9-ef954ff4c199","Type":"ContainerDied","Data":"5b258586b283b24194daefae3c5ff0b82a10a4d6dc595f1d100d7515572ddbff"}
Jul 17 18:16:48.408812 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:48.408565    2572 scope.go:117] "RemoveContainer" containerID="c5b5e431986a4dd76e1f5715e73eeed8c53432c8f26ff36532e1ca176bdccdce"
Jul 17 18:16:48.408889 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:48.408872    2572 scope.go:117] "RemoveContainer" containerID="5b258586b283b24194daefae3c5ff0b82a10a4d6dc595f1d100d7515572ddbff"
Jul 17 18:16:48.409136 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:48.409118    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt_odh-ai-gateway-infra(55dc52e6-4802-4fb2-9fe9-ef954ff4c199)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199"
Jul 17 18:16:49.438471 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:49.438418    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"]
Jul 17 18:16:49.587214 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:49.587188    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"
Jul 17 18:16:49.671693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:49.671643    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dc52e6-4802-4fb2-9fe9-ef954ff4c199-maas-api-service-ca\") pod \"55dc52e6-4802-4fb2-9fe9-ef954ff4c199\" (UID: \"55dc52e6-4802-4fb2-9fe9-ef954ff4c199\") "
Jul 17 18:16:49.672043 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:49.672021    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55dc52e6-4802-4fb2-9fe9-ef954ff4c199-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55dc52e6-4802-4fb2-9fe9-ef954ff4c199" (UID: "55dc52e6-4802-4fb2-9fe9-ef954ff4c199"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:16:49.772694 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:49.772598    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dc52e6-4802-4fb2-9fe9-ef954ff4c199-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:16:50.422391 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:50.422354    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"
Jul 17 18:16:50.422391 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:50.422371    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt" event={"ID":"55dc52e6-4802-4fb2-9fe9-ef954ff4c199","Type":"ContainerDied","Data":"d228a4f4609bd3297653e87df77fec78e45e99237341d5f126da631f9dda08bc"}
Jul 17 18:16:50.422686 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:50.422474    2572 scope.go:117] "RemoveContainer" containerID="5b258586b283b24194daefae3c5ff0b82a10a4d6dc595f1d100d7515572ddbff"
Jul 17 18:16:50.449697 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:50.449658    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"]
Jul 17 18:16:50.457310 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:50.457277    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-95jmt"]
Jul 17 18:16:50.515992 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:50.515959    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" path="/var/lib/kubelet/pods/55dc52e6-4802-4fb2-9fe9-ef954ff4c199/volumes"
Jul 17 18:16:52.510083 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:52.510041    2572 scope.go:117] "RemoveContainer" containerID="34afea826a4917b9514198a6b57becb6776df759eb966db778527739d45cd615"
Jul 17 18:16:53.443623 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:53.443584    2572 generic.go:356] "Generic (PLEG): container finished" podID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerID="4a7995b4ebd84c1a99947dcba442a87d600e3a6b05583463ad9dd8b01d6f1580" exitCode=6
Jul 17 18:16:53.443818 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:53.443651    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb" event={"ID":"f92fccef-e0c2-49e7-a5fd-919ea8c404a7","Type":"ContainerDied","Data":"4a7995b4ebd84c1a99947dcba442a87d600e3a6b05583463ad9dd8b01d6f1580"}
Jul 17 18:16:53.443818 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:53.443718    2572 scope.go:117] "RemoveContainer" containerID="34afea826a4917b9514198a6b57becb6776df759eb966db778527739d45cd615"
Jul 17 18:16:53.444078 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:53.444043    2572 scope.go:117] "RemoveContainer" containerID="4a7995b4ebd84c1a99947dcba442a87d600e3a6b05583463ad9dd8b01d6f1580"
Jul 17 18:16:53.444273 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:16:53.444249    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-r98kb_odh-ai-gateway-infra(f92fccef-e0c2-49e7-a5fd-919ea8c404a7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7"
Jul 17 18:16:54.471274 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:54.471231    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"]
Jul 17 18:16:54.612407 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:54.612377    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"
Jul 17 18:16:54.720633 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:54.720596    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92fccef-e0c2-49e7-a5fd-919ea8c404a7-maas-api-service-ca\") pod \"f92fccef-e0c2-49e7-a5fd-919ea8c404a7\" (UID: \"f92fccef-e0c2-49e7-a5fd-919ea8c404a7\") "
Jul 17 18:16:54.720992 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:54.720972    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f92fccef-e0c2-49e7-a5fd-919ea8c404a7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f92fccef-e0c2-49e7-a5fd-919ea8c404a7" (UID: "f92fccef-e0c2-49e7-a5fd-919ea8c404a7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:16:54.821751 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:54.821714    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92fccef-e0c2-49e7-a5fd-919ea8c404a7-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:16:55.458201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:55.458164    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"
Jul 17 18:16:55.458201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:55.458195    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb" event={"ID":"f92fccef-e0c2-49e7-a5fd-919ea8c404a7","Type":"ContainerDied","Data":"e24f28ff6f542fc828eefb6e9967acb0b294be06396d4e99b6a3a1bb35eb8c39"}
Jul 17 18:16:55.458535 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:55.458258    2572 scope.go:117] "RemoveContainer" containerID="4a7995b4ebd84c1a99947dcba442a87d600e3a6b05583463ad9dd8b01d6f1580"
Jul 17 18:16:55.479265 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:55.479223    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"]
Jul 17 18:16:55.487733 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:55.487695    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-r98kb"]
Jul 17 18:16:56.516068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:16:56.516028    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" path="/var/lib/kubelet/pods/f92fccef-e0c2-49e7-a5fd-919ea8c404a7/volumes"
Jul 17 18:17:04.769897 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.769860    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"]
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770193    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770203    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770214    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae69940f-ba1d-4017-962c-1633a020553a" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770220    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae69940f-ba1d-4017-962c-1633a020553a" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770226    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770231    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770238    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae69940f-ba1d-4017-962c-1633a020553a" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770242    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae69940f-ba1d-4017-962c-1633a020553a" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770258    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770262    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770269    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770274    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd79e45-a98b-4419-b3c1-3ec1068d05fb" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770280    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770284    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770292    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770296    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770303    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerName="revoke-keys"
Jul 17 18:17:04.770304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770308    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerName="revoke-keys"
Jul 17 18:17:04.770873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770362    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerName="revoke-keys"
Jul 17 18:17:04.770873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770370    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae69940f-ba1d-4017-962c-1633a020553a" containerName="revoke-keys"
Jul 17 18:17:04.770873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770377    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerName="revoke-keys"
Jul 17 18:17:04.770873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770383    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae69940f-ba1d-4017-962c-1633a020553a" containerName="revoke-keys"
Jul 17 18:17:04.770873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770388    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerName="revoke-keys"
Jul 17 18:17:04.770873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770393    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f92fccef-e0c2-49e7-a5fd-919ea8c404a7" containerName="revoke-keys"
Jul 17 18:17:04.770873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770398    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerName="revoke-keys"
Jul 17 18:17:04.770873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.770411    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="55dc52e6-4802-4fb2-9fe9-ef954ff4c199" containerName="revoke-keys"
Jul 17 18:17:04.773554 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.773532    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"
Jul 17 18:17:04.776379 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.776348    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:17:04.776379 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.776348    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:17:04.781238 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.781194    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"]
Jul 17 18:17:04.913924 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:04.913872    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9e4752-affe-4017-8ab6-3129c97506ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz\" (UID: \"aa9e4752-affe-4017-8ab6-3129c97506ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"
Jul 17 18:17:05.015433 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:05.015387    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9e4752-affe-4017-8ab6-3129c97506ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz\" (UID: \"aa9e4752-affe-4017-8ab6-3129c97506ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"
Jul 17 18:17:05.016095 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:05.016073    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9e4752-affe-4017-8ab6-3129c97506ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz\" (UID: \"aa9e4752-affe-4017-8ab6-3129c97506ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"
Jul 17 18:17:05.086519 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:05.086457    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"
Jul 17 18:17:05.224876 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:05.224840    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"]
Jul 17 18:17:05.226529 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:17:05.226469    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9e4752_affe_4017_8ab6_3129c97506ef.slice/crio-e842ace07d9e6c84342466c63ff545c5ff1813c9b5e13cd512ef934d3874de75 WatchSource:0}: Error finding container e842ace07d9e6c84342466c63ff545c5ff1813c9b5e13cd512ef934d3874de75: Status 404 returned error can't find the container with id e842ace07d9e6c84342466c63ff545c5ff1813c9b5e13cd512ef934d3874de75
Jul 17 18:17:05.520572 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:05.520535    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz" event={"ID":"aa9e4752-affe-4017-8ab6-3129c97506ef","Type":"ContainerStarted","Data":"e842ace07d9e6c84342466c63ff545c5ff1813c9b5e13cd512ef934d3874de75"}
Jul 17 18:17:06.528314 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:06.528275    2572 generic.go:356] "Generic (PLEG): container finished" podID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerID="df399121a352ae29e20442ffbb519ab63afef78ebad3776aecd894a8c8420a88" exitCode=6
Jul 17 18:17:06.528314 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:06.528319    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz" event={"ID":"aa9e4752-affe-4017-8ab6-3129c97506ef","Type":"ContainerDied","Data":"df399121a352ae29e20442ffbb519ab63afef78ebad3776aecd894a8c8420a88"}
Jul 17 18:17:06.528786 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:06.528574    2572 scope.go:117] "RemoveContainer" containerID="df399121a352ae29e20442ffbb519ab63afef78ebad3776aecd894a8c8420a88"
Jul 17 18:17:07.535871 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:07.535767    2572 generic.go:356] "Generic (PLEG): container finished" podID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerID="70fd7ace880417a19175c3bb0a6d2880425ce985d01f609648bdeacda8228834" exitCode=6
Jul 17 18:17:07.536383 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:07.535908    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz" event={"ID":"aa9e4752-affe-4017-8ab6-3129c97506ef","Type":"ContainerDied","Data":"70fd7ace880417a19175c3bb0a6d2880425ce985d01f609648bdeacda8228834"}
Jul 17 18:17:07.536383 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:07.535974    2572 scope.go:117] "RemoveContainer" containerID="df399121a352ae29e20442ffbb519ab63afef78ebad3776aecd894a8c8420a88"
Jul 17 18:17:07.536383 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:07.536182    2572 scope.go:117] "RemoveContainer" containerID="70fd7ace880417a19175c3bb0a6d2880425ce985d01f609648bdeacda8228834"
Jul 17 18:17:07.536573 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:07.536394    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz_odh-ai-gateway-infra(aa9e4752-affe-4017-8ab6-3129c97506ef)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef"
Jul 17 18:17:08.544384 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:08.544352    2572 scope.go:117] "RemoveContainer" containerID="70fd7ace880417a19175c3bb0a6d2880425ce985d01f609648bdeacda8228834"
Jul 17 18:17:08.544855 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:08.544607    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz_odh-ai-gateway-infra(aa9e4752-affe-4017-8ab6-3129c97506ef)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef"
Jul 17 18:17:19.510480 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:19.510445    2572 scope.go:117] "RemoveContainer" containerID="70fd7ace880417a19175c3bb0a6d2880425ce985d01f609648bdeacda8228834"
Jul 17 18:17:20.552707 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.552617    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"]
Jul 17 18:17:20.553098 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.552968    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae69940f-ba1d-4017-962c-1633a020553a" containerName="revoke-keys"
Jul 17 18:17:20.553098 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.552979    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae69940f-ba1d-4017-962c-1633a020553a" containerName="revoke-keys"
Jul 17 18:17:20.553098 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.553048    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae69940f-ba1d-4017-962c-1633a020553a" containerName="revoke-keys"
Jul 17 18:17:20.556087 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.556066    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"
Jul 17 18:17:20.564689 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.564658    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"]
Jul 17 18:17:20.615547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.615482    2572 generic.go:356] "Generic (PLEG): container finished" podID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerID="f99d6944f94224e67e03a943b7b43e02a24915f30a4aa19190595994097ce213" exitCode=6
Jul 17 18:17:20.615740 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.615550    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz" event={"ID":"aa9e4752-affe-4017-8ab6-3129c97506ef","Type":"ContainerDied","Data":"f99d6944f94224e67e03a943b7b43e02a24915f30a4aa19190595994097ce213"}
Jul 17 18:17:20.615740 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.615614    2572 scope.go:117] "RemoveContainer" containerID="70fd7ace880417a19175c3bb0a6d2880425ce985d01f609648bdeacda8228834"
Jul 17 18:17:20.615964 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.615943    2572 scope.go:117] "RemoveContainer" containerID="f99d6944f94224e67e03a943b7b43e02a24915f30a4aa19190595994097ce213"
Jul 17 18:17:20.616157 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:20.616136    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz_odh-ai-gateway-infra(aa9e4752-affe-4017-8ab6-3129c97506ef)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef"
Jul 17 18:17:20.663990 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.663945    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5170e57-9eca-4bcd-9a6f-7108da612974-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4\" (UID: \"c5170e57-9eca-4bcd-9a6f-7108da612974\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"
Jul 17 18:17:20.765308 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.765272    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5170e57-9eca-4bcd-9a6f-7108da612974-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4\" (UID: \"c5170e57-9eca-4bcd-9a6f-7108da612974\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"
Jul 17 18:17:20.765895 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.765876    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5170e57-9eca-4bcd-9a6f-7108da612974-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4\" (UID: \"c5170e57-9eca-4bcd-9a6f-7108da612974\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"
Jul 17 18:17:20.868841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:20.868807    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"
Jul 17 18:17:21.009137 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:21.009109    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"]
Jul 17 18:17:21.010785 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:17:21.010753    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5170e57_9eca_4bcd_9a6f_7108da612974.slice/crio-bf0b427a79084132c2e53e9e7ee7336d378c758fd47ff3c8e7c8e0de74c52363 WatchSource:0}: Error finding container bf0b427a79084132c2e53e9e7ee7336d378c758fd47ff3c8e7c8e0de74c52363: Status 404 returned error can't find the container with id bf0b427a79084132c2e53e9e7ee7336d378c758fd47ff3c8e7c8e0de74c52363
Jul 17 18:17:21.633575 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:21.633534    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" event={"ID":"c5170e57-9eca-4bcd-9a6f-7108da612974","Type":"ContainerStarted","Data":"f7bccf46e106bd0397341d5a6b7e282c543deb4fbd98ecac659ca5dce443d225"}
Jul 17 18:17:21.634065 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:21.633619    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" event={"ID":"c5170e57-9eca-4bcd-9a6f-7108da612974","Type":"ContainerStarted","Data":"bf0b427a79084132c2e53e9e7ee7336d378c758fd47ff3c8e7c8e0de74c52363"}
Jul 17 18:17:21.642685 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:21.642652    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"]
Jul 17 18:17:21.653727 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:21.653671    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" podStartSLOduration=1.6536505400000001 podStartE2EDuration="1.65365054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:17:21.650780483 +0000 UTC m=+3213.656136349" watchObservedRunningTime="2026-07-17 18:17:21.65365054 +0000 UTC m=+3213.659006459"
Jul 17 18:17:21.806391 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:21.806361    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"
Jul 17 18:17:21.975112 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:21.975071    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9e4752-affe-4017-8ab6-3129c97506ef-maas-api-service-ca\") pod \"aa9e4752-affe-4017-8ab6-3129c97506ef\" (UID: \"aa9e4752-affe-4017-8ab6-3129c97506ef\") "
Jul 17 18:17:21.975520 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:21.975468    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa9e4752-affe-4017-8ab6-3129c97506ef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aa9e4752-affe-4017-8ab6-3129c97506ef" (UID: "aa9e4752-affe-4017-8ab6-3129c97506ef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:17:22.075983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:22.075940    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9e4752-affe-4017-8ab6-3129c97506ef-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:22.640943 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:22.640906    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"
Jul 17 18:17:22.641432 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:22.640943    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz" event={"ID":"aa9e4752-affe-4017-8ab6-3129c97506ef","Type":"ContainerDied","Data":"e842ace07d9e6c84342466c63ff545c5ff1813c9b5e13cd512ef934d3874de75"}
Jul 17 18:17:22.641432 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:22.641021    2572 scope.go:117] "RemoveContainer" containerID="f99d6944f94224e67e03a943b7b43e02a24915f30a4aa19190595994097ce213"
Jul 17 18:17:22.642788 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:22.642756    2572 generic.go:356] "Generic (PLEG): container finished" podID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerID="f7bccf46e106bd0397341d5a6b7e282c543deb4fbd98ecac659ca5dce443d225" exitCode=6
Jul 17 18:17:22.642931 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:22.642807    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" event={"ID":"c5170e57-9eca-4bcd-9a6f-7108da612974","Type":"ContainerDied","Data":"f7bccf46e106bd0397341d5a6b7e282c543deb4fbd98ecac659ca5dce443d225"}
Jul 17 18:17:22.643097 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:22.643079    2572 scope.go:117] "RemoveContainer" containerID="f7bccf46e106bd0397341d5a6b7e282c543deb4fbd98ecac659ca5dce443d225"
Jul 17 18:17:22.673544 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:22.673483    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"]
Jul 17 18:17:22.677702 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:22.677674    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-2s2vz"]
Jul 17 18:17:23.651582 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:23.651457    2572 generic.go:356] "Generic (PLEG): container finished" podID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerID="b66ce61b85d1af82e68ab649e2c5f1fdfa9b23fe7438ff4bbc7d699e3e8e9ddd" exitCode=6
Jul 17 18:17:23.651582 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:23.651544    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" event={"ID":"c5170e57-9eca-4bcd-9a6f-7108da612974","Type":"ContainerDied","Data":"b66ce61b85d1af82e68ab649e2c5f1fdfa9b23fe7438ff4bbc7d699e3e8e9ddd"}
Jul 17 18:17:23.652049 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:23.651601    2572 scope.go:117] "RemoveContainer" containerID="f7bccf46e106bd0397341d5a6b7e282c543deb4fbd98ecac659ca5dce443d225"
Jul 17 18:17:23.652049 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:23.651797    2572 scope.go:117] "RemoveContainer" containerID="b66ce61b85d1af82e68ab649e2c5f1fdfa9b23fe7438ff4bbc7d699e3e8e9ddd"
Jul 17 18:17:23.652049 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:23.652021    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4_odh-ai-gateway-infra(c5170e57-9eca-4bcd-9a6f-7108da612974)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974"
Jul 17 18:17:24.515623 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:24.515583    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" path="/var/lib/kubelet/pods/aa9e4752-affe-4017-8ab6-3129c97506ef/volumes"
Jul 17 18:17:24.659229 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:24.659202    2572 scope.go:117] "RemoveContainer" containerID="b66ce61b85d1af82e68ab649e2c5f1fdfa9b23fe7438ff4bbc7d699e3e8e9ddd"
Jul 17 18:17:24.659641 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:24.659407    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4_odh-ai-gateway-infra(c5170e57-9eca-4bcd-9a6f-7108da612974)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974"
Jul 17 18:17:25.075219 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:25.075183    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"]
Jul 17 18:17:27.011520 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.011469    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"]
Jul 17 18:17:27.011938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.011829    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerName="revoke-keys"
Jul 17 18:17:27.011938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.011841    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerName="revoke-keys"
Jul 17 18:17:27.011938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.011852    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerName="revoke-keys"
Jul 17 18:17:27.011938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.011858    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerName="revoke-keys"
Jul 17 18:17:27.011938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.011869    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerName="revoke-keys"
Jul 17 18:17:27.011938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.011875    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerName="revoke-keys"
Jul 17 18:17:27.011938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.011938    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerName="revoke-keys"
Jul 17 18:17:27.012199 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.011947    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerName="revoke-keys"
Jul 17 18:17:27.016503 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.016464    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"
Jul 17 18:17:27.025233 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.025185    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"]
Jul 17 18:17:27.124905 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.124856    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f157795d-0b0f-4083-b723-18f8afb831f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-57vdn\" (UID: \"f157795d-0b0f-4083-b723-18f8afb831f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"
Jul 17 18:17:27.226441 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.226383    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f157795d-0b0f-4083-b723-18f8afb831f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-57vdn\" (UID: \"f157795d-0b0f-4083-b723-18f8afb831f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"
Jul 17 18:17:27.227044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.227021    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f157795d-0b0f-4083-b723-18f8afb831f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-57vdn\" (UID: \"f157795d-0b0f-4083-b723-18f8afb831f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"
Jul 17 18:17:27.335603 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.335569    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"
Jul 17 18:17:27.478304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.478270    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"]
Jul 17 18:17:27.479685 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:17:27.479655    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf157795d_0b0f_4083_b723_18f8afb831f5.slice/crio-ceba2b4faa1d953ca09398a3110b05aaae689ea3d942e4bb5dc3bc51bb906430 WatchSource:0}: Error finding container ceba2b4faa1d953ca09398a3110b05aaae689ea3d942e4bb5dc3bc51bb906430: Status 404 returned error can't find the container with id ceba2b4faa1d953ca09398a3110b05aaae689ea3d942e4bb5dc3bc51bb906430
Jul 17 18:17:27.676899 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:27.676860    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn" event={"ID":"f157795d-0b0f-4083-b723-18f8afb831f5","Type":"ContainerStarted","Data":"ceba2b4faa1d953ca09398a3110b05aaae689ea3d942e4bb5dc3bc51bb906430"}
Jul 17 18:17:28.683983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:28.683937    2572 generic.go:356] "Generic (PLEG): container finished" podID="f157795d-0b0f-4083-b723-18f8afb831f5" containerID="de26a4abf6482436b719add1034e7370ac65e9705b8c84e7c2fe9ed779c10447" exitCode=6
Jul 17 18:17:28.684483 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:28.684019    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn" event={"ID":"f157795d-0b0f-4083-b723-18f8afb831f5","Type":"ContainerDied","Data":"de26a4abf6482436b719add1034e7370ac65e9705b8c84e7c2fe9ed779c10447"}
Jul 17 18:17:28.684483 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:28.684221    2572 scope.go:117] "RemoveContainer" containerID="de26a4abf6482436b719add1034e7370ac65e9705b8c84e7c2fe9ed779c10447"
Jul 17 18:17:28.915578 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:28.915530    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"]
Jul 17 18:17:28.916038 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:28.916018    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa9e4752-affe-4017-8ab6-3129c97506ef" containerName="revoke-keys"
Jul 17 18:17:28.920519 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:28.920462    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"
Jul 17 18:17:28.926716 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:28.926683    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"]
Jul 17 18:17:28.940529 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:28.940482    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a56407-0d7f-49e2-bc70-676624abbea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5\" (UID: \"43a56407-0d7f-49e2-bc70-676624abbea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"
Jul 17 18:17:29.041792 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.041763    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a56407-0d7f-49e2-bc70-676624abbea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5\" (UID: \"43a56407-0d7f-49e2-bc70-676624abbea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"
Jul 17 18:17:29.042343 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.042323    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a56407-0d7f-49e2-bc70-676624abbea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5\" (UID: \"43a56407-0d7f-49e2-bc70-676624abbea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"
Jul 17 18:17:29.239864 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.239762    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"
Jul 17 18:17:29.379305 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.379280    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"]
Jul 17 18:17:29.381145 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:17:29.381117    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a56407_0d7f_49e2_bc70_676624abbea7.slice/crio-b49708adf288a8291023c3af992903bc6be3e8dc21796db9b3dd7020f36f6b9e WatchSource:0}: Error finding container b49708adf288a8291023c3af992903bc6be3e8dc21796db9b3dd7020f36f6b9e: Status 404 returned error can't find the container with id b49708adf288a8291023c3af992903bc6be3e8dc21796db9b3dd7020f36f6b9e
Jul 17 18:17:29.691471 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.691441    2572 generic.go:356] "Generic (PLEG): container finished" podID="f157795d-0b0f-4083-b723-18f8afb831f5" containerID="a387bc8dc0be7d8fec66d7065cfbc1d56976fe7bc58cac23e7efe1bbfabddc38" exitCode=6
Jul 17 18:17:29.691963 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.691523    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn" event={"ID":"f157795d-0b0f-4083-b723-18f8afb831f5","Type":"ContainerDied","Data":"a387bc8dc0be7d8fec66d7065cfbc1d56976fe7bc58cac23e7efe1bbfabddc38"}
Jul 17 18:17:29.691963 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.691586    2572 scope.go:117] "RemoveContainer" containerID="de26a4abf6482436b719add1034e7370ac65e9705b8c84e7c2fe9ed779c10447"
Jul 17 18:17:29.691963 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.691798    2572 scope.go:117] "RemoveContainer" containerID="a387bc8dc0be7d8fec66d7065cfbc1d56976fe7bc58cac23e7efe1bbfabddc38"
Jul 17 18:17:29.692093 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:29.692044    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-57vdn_odh-ai-gateway-infra(f157795d-0b0f-4083-b723-18f8afb831f5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn" podUID="f157795d-0b0f-4083-b723-18f8afb831f5"
Jul 17 18:17:29.693331 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.693308    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" event={"ID":"43a56407-0d7f-49e2-bc70-676624abbea7","Type":"ContainerStarted","Data":"77ac3f3d5f23053ea09e3512c59f4d042b63d38c721b4e8607262fb18c59e119"}
Jul 17 18:17:29.693446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.693355    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" event={"ID":"43a56407-0d7f-49e2-bc70-676624abbea7","Type":"ContainerStarted","Data":"b49708adf288a8291023c3af992903bc6be3e8dc21796db9b3dd7020f36f6b9e"}
Jul 17 18:17:29.725923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:29.725838    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" podStartSLOduration=1.725814732 podStartE2EDuration="1.725814732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:17:29.722344476 +0000 UTC m=+3221.727700331" watchObservedRunningTime="2026-07-17 18:17:29.725814732 +0000 UTC m=+3221.731170587"
Jul 17 18:17:30.700854 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:30.700821    2572 scope.go:117] "RemoveContainer" containerID="a387bc8dc0be7d8fec66d7065cfbc1d56976fe7bc58cac23e7efe1bbfabddc38"
Jul 17 18:17:30.701277 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:30.701044    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-57vdn_odh-ai-gateway-infra(f157795d-0b0f-4083-b723-18f8afb831f5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn" podUID="f157795d-0b0f-4083-b723-18f8afb831f5"
Jul 17 18:17:30.702118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:30.702092    2572 generic.go:356] "Generic (PLEG): container finished" podID="43a56407-0d7f-49e2-bc70-676624abbea7" containerID="77ac3f3d5f23053ea09e3512c59f4d042b63d38c721b4e8607262fb18c59e119" exitCode=6
Jul 17 18:17:30.702220 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:30.702152    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" event={"ID":"43a56407-0d7f-49e2-bc70-676624abbea7","Type":"ContainerDied","Data":"77ac3f3d5f23053ea09e3512c59f4d042b63d38c721b4e8607262fb18c59e119"}
Jul 17 18:17:30.702349 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:30.702338    2572 scope.go:117] "RemoveContainer" containerID="77ac3f3d5f23053ea09e3512c59f4d042b63d38c721b4e8607262fb18c59e119"
Jul 17 18:17:31.709367 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:31.709267    2572 generic.go:356] "Generic (PLEG): container finished" podID="43a56407-0d7f-49e2-bc70-676624abbea7" containerID="851888ed9bb55d1740998da755683819500425e7e53fa53f624705481eda3a12" exitCode=6
Jul 17 18:17:31.709367 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:31.709349    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" event={"ID":"43a56407-0d7f-49e2-bc70-676624abbea7","Type":"ContainerDied","Data":"851888ed9bb55d1740998da755683819500425e7e53fa53f624705481eda3a12"}
Jul 17 18:17:31.709918 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:31.709445    2572 scope.go:117] "RemoveContainer" containerID="77ac3f3d5f23053ea09e3512c59f4d042b63d38c721b4e8607262fb18c59e119"
Jul 17 18:17:31.709918 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:31.709715    2572 scope.go:117] "RemoveContainer" containerID="851888ed9bb55d1740998da755683819500425e7e53fa53f624705481eda3a12"
Jul 17 18:17:31.710017 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:31.709997    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5_odh-ai-gateway-infra(43a56407-0d7f-49e2-bc70-676624abbea7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" podUID="43a56407-0d7f-49e2-bc70-676624abbea7"
Jul 17 18:17:32.716854 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:32.716758    2572 scope.go:117] "RemoveContainer" containerID="851888ed9bb55d1740998da755683819500425e7e53fa53f624705481eda3a12"
Jul 17 18:17:32.717327 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:32.717019    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5_odh-ai-gateway-infra(43a56407-0d7f-49e2-bc70-676624abbea7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" podUID="43a56407-0d7f-49e2-bc70-676624abbea7"
Jul 17 18:17:35.510264 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:35.510227    2572 scope.go:117] "RemoveContainer" containerID="b66ce61b85d1af82e68ab649e2c5f1fdfa9b23fe7438ff4bbc7d699e3e8e9ddd"
Jul 17 18:17:36.742025 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:36.741985    2572 generic.go:356] "Generic (PLEG): container finished" podID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerID="36267557f17d824643ae879b06bc4d32158e21b0bf405fd8c5b91f34c11f858f" exitCode=6
Jul 17 18:17:36.742594 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:36.742055    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" event={"ID":"c5170e57-9eca-4bcd-9a6f-7108da612974","Type":"ContainerDied","Data":"36267557f17d824643ae879b06bc4d32158e21b0bf405fd8c5b91f34c11f858f"}
Jul 17 18:17:36.742594 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:36.742120    2572 scope.go:117] "RemoveContainer" containerID="b66ce61b85d1af82e68ab649e2c5f1fdfa9b23fe7438ff4bbc7d699e3e8e9ddd"
Jul 17 18:17:36.742594 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:36.742531    2572 scope.go:117] "RemoveContainer" containerID="36267557f17d824643ae879b06bc4d32158e21b0bf405fd8c5b91f34c11f858f"
Jul 17 18:17:36.742824 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:36.742802    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4_odh-ai-gateway-infra(c5170e57-9eca-4bcd-9a6f-7108da612974)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974"
Jul 17 18:17:37.773352 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:37.773310    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"]
Jul 17 18:17:37.915384 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:37.915357    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"
Jul 17 18:17:38.018092 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:38.018052    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5170e57-9eca-4bcd-9a6f-7108da612974-maas-api-service-ca\") pod \"c5170e57-9eca-4bcd-9a6f-7108da612974\" (UID: \"c5170e57-9eca-4bcd-9a6f-7108da612974\") "
Jul 17 18:17:38.018470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:38.018423    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5170e57-9eca-4bcd-9a6f-7108da612974-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5170e57-9eca-4bcd-9a6f-7108da612974" (UID: "c5170e57-9eca-4bcd-9a6f-7108da612974"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:17:38.118883 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:38.118843    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5170e57-9eca-4bcd-9a6f-7108da612974-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:38.763033 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:38.763001    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"
Jul 17 18:17:38.763228 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:38.763038    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4" event={"ID":"c5170e57-9eca-4bcd-9a6f-7108da612974","Type":"ContainerDied","Data":"bf0b427a79084132c2e53e9e7ee7336d378c758fd47ff3c8e7c8e0de74c52363"}
Jul 17 18:17:38.763228 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:38.763105    2572 scope.go:117] "RemoveContainer" containerID="36267557f17d824643ae879b06bc4d32158e21b0bf405fd8c5b91f34c11f858f"
Jul 17 18:17:38.784097 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:38.784018    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"]
Jul 17 18:17:38.791004 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:38.790970    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wt2b4"]
Jul 17 18:17:40.095447 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.095382    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472" podUID="49b8ba94-5cc8-4a8d-b7a4-a72aad654786" containerName="main" containerID="cri-o://6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d" gracePeriod=45
Jul 17 18:17:40.356409 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.356339    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:17:40.440304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440202    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tls-certs\") pod \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") "
Jul 17 18:17:40.440304 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440283    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-home\") pod \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") "
Jul 17 18:17:40.440589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440323    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8gsg\" (UniqueName: \"kubernetes.io/projected/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-kube-api-access-r8gsg\") pod \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") "
Jul 17 18:17:40.440589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440344    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-model-cache\") pod \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") "
Jul 17 18:17:40.440589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440404    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-dshm\") pod \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") "
Jul 17 18:17:40.440589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440436    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tmp-dir\") pod \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\" (UID: \"49b8ba94-5cc8-4a8d-b7a4-a72aad654786\") "
Jul 17 18:17:40.440801 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440625    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-home" (OuterVolumeSpecName: "home") pod "49b8ba94-5cc8-4a8d-b7a4-a72aad654786" (UID: "49b8ba94-5cc8-4a8d-b7a4-a72aad654786"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:17:40.440801 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440730    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "49b8ba94-5cc8-4a8d-b7a4-a72aad654786" (UID: "49b8ba94-5cc8-4a8d-b7a4-a72aad654786"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:17:40.440801 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440744    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-model-cache" (OuterVolumeSpecName: "model-cache") pod "49b8ba94-5cc8-4a8d-b7a4-a72aad654786" (UID: "49b8ba94-5cc8-4a8d-b7a4-a72aad654786"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:17:40.440801 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.440760    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-home\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:40.442664 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.442629    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-dshm" (OuterVolumeSpecName: "dshm") pod "49b8ba94-5cc8-4a8d-b7a4-a72aad654786" (UID: "49b8ba94-5cc8-4a8d-b7a4-a72aad654786"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:17:40.442773 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.442678    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "49b8ba94-5cc8-4a8d-b7a4-a72aad654786" (UID: "49b8ba94-5cc8-4a8d-b7a4-a72aad654786"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:17:40.442773 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.442702    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-kube-api-access-r8gsg" (OuterVolumeSpecName: "kube-api-access-r8gsg") pod "49b8ba94-5cc8-4a8d-b7a4-a72aad654786" (UID: "49b8ba94-5cc8-4a8d-b7a4-a72aad654786"). InnerVolumeSpecName "kube-api-access-r8gsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:17:40.516560 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.516522    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" path="/var/lib/kubelet/pods/c5170e57-9eca-4bcd-9a6f-7108da612974/volumes"
Jul 17 18:17:40.541201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.541154    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-dshm\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:40.541201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.541191    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tmp-dir\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:40.541201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.541200    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-tls-certs\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:40.541201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.541212    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8gsg\" (UniqueName: \"kubernetes.io/projected/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-kube-api-access-r8gsg\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:40.541483 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.541221    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b8ba94-5cc8-4a8d-b7a4-a72aad654786-model-cache\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:40.778320 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.778227    2572 generic.go:356] "Generic (PLEG): container finished" podID="49b8ba94-5cc8-4a8d-b7a4-a72aad654786" containerID="6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d" exitCode=0
Jul 17 18:17:40.778320 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.778275    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"
Jul 17 18:17:40.778590 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.778311    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472" event={"ID":"49b8ba94-5cc8-4a8d-b7a4-a72aad654786","Type":"ContainerDied","Data":"6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d"}
Jul 17 18:17:40.778590 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.778358    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472" event={"ID":"49b8ba94-5cc8-4a8d-b7a4-a72aad654786","Type":"ContainerDied","Data":"7a7e386029646b9a84c680c2cd609da8e3dc83909f2db7bbdd6d1f85b60ac01f"}
Jul 17 18:17:40.778590 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.778378    2572 scope.go:117] "RemoveContainer" containerID="6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d"
Jul 17 18:17:40.787678 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.787642    2572 scope.go:117] "RemoveContainer" containerID="6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d"
Jul 17 18:17:40.787970 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:40.787948    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d\": container with ID starting with 6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d not found: ID does not exist" containerID="6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d"
Jul 17 18:17:40.788020 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.787983    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d"} err="failed to get container status \"6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d\": rpc error: code = NotFound desc = could not find container \"6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d\": container with ID starting with 6c2e8468142c7a239bb673d7ae41a86008acac153d97b77fecddaea98cdb498d not found: ID does not exist"
Jul 17 18:17:40.796630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.796600    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"]
Jul 17 18:17:40.802044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:40.802018    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-22fb10/auth-test-model-22fb10-kserve-84994fd847-d5472"]
Jul 17 18:17:41.510182 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:41.510147    2572 scope.go:117] "RemoveContainer" containerID="a387bc8dc0be7d8fec66d7065cfbc1d56976fe7bc58cac23e7efe1bbfabddc38"
Jul 17 18:17:42.515469 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:42.515379    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49b8ba94-5cc8-4a8d-b7a4-a72aad654786" path="/var/lib/kubelet/pods/49b8ba94-5cc8-4a8d-b7a4-a72aad654786/volumes"
Jul 17 18:17:42.793941 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:42.793846    2572 generic.go:356] "Generic (PLEG): container finished" podID="f157795d-0b0f-4083-b723-18f8afb831f5" containerID="b13fd7559a830bdfafdf160cdf8d9af3fd359ed9f522904a1f8479b672711ed5" exitCode=6
Jul 17 18:17:42.793941 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:42.793904    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn" event={"ID":"f157795d-0b0f-4083-b723-18f8afb831f5","Type":"ContainerDied","Data":"b13fd7559a830bdfafdf160cdf8d9af3fd359ed9f522904a1f8479b672711ed5"}
Jul 17 18:17:42.794140 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:42.793981    2572 scope.go:117] "RemoveContainer" containerID="a387bc8dc0be7d8fec66d7065cfbc1d56976fe7bc58cac23e7efe1bbfabddc38"
Jul 17 18:17:42.794315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:42.794283    2572 scope.go:117] "RemoveContainer" containerID="b13fd7559a830bdfafdf160cdf8d9af3fd359ed9f522904a1f8479b672711ed5"
Jul 17 18:17:42.794624 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:42.794601    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-57vdn_odh-ai-gateway-infra(f157795d-0b0f-4083-b723-18f8afb831f5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn" podUID="f157795d-0b0f-4083-b723-18f8afb831f5"
Jul 17 18:17:43.830575 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:43.830538    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"]
Jul 17 18:17:43.972663 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:43.972632    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"
Jul 17 18:17:44.073096 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:44.073055    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f157795d-0b0f-4083-b723-18f8afb831f5-maas-api-service-ca\") pod \"f157795d-0b0f-4083-b723-18f8afb831f5\" (UID: \"f157795d-0b0f-4083-b723-18f8afb831f5\") "
Jul 17 18:17:44.073425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:44.073400    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f157795d-0b0f-4083-b723-18f8afb831f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f157795d-0b0f-4083-b723-18f8afb831f5" (UID: "f157795d-0b0f-4083-b723-18f8afb831f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:17:44.173859 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:44.173766    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f157795d-0b0f-4083-b723-18f8afb831f5-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:44.809288 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:44.809250    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn" event={"ID":"f157795d-0b0f-4083-b723-18f8afb831f5","Type":"ContainerDied","Data":"ceba2b4faa1d953ca09398a3110b05aaae689ea3d942e4bb5dc3bc51bb906430"}
Jul 17 18:17:44.809288 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:44.809267    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"
Jul 17 18:17:44.809548 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:44.809310    2572 scope.go:117] "RemoveContainer" containerID="b13fd7559a830bdfafdf160cdf8d9af3fd359ed9f522904a1f8479b672711ed5"
Jul 17 18:17:44.829650 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:44.829613    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"]
Jul 17 18:17:44.836013 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:44.835977    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-57vdn"]
Jul 17 18:17:46.516238 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:46.516207    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" path="/var/lib/kubelet/pods/f157795d-0b0f-4083-b723-18f8afb831f5/volumes"
Jul 17 18:17:47.509952 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:47.509918    2572 scope.go:117] "RemoveContainer" containerID="851888ed9bb55d1740998da755683819500425e7e53fa53f624705481eda3a12"
Jul 17 18:17:47.828954 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:47.828919    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" event={"ID":"43a56407-0d7f-49e2-bc70-676624abbea7","Type":"ContainerStarted","Data":"10cb022702c5754dbf2cd4dbfa44edd5d300ed6b9004227bbb72f6013c0095ef"}
Jul 17 18:17:48.836873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:48.836838    2572 generic.go:356] "Generic (PLEG): container finished" podID="43a56407-0d7f-49e2-bc70-676624abbea7" containerID="10cb022702c5754dbf2cd4dbfa44edd5d300ed6b9004227bbb72f6013c0095ef" exitCode=6
Jul 17 18:17:48.837321 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:48.836907    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" event={"ID":"43a56407-0d7f-49e2-bc70-676624abbea7","Type":"ContainerDied","Data":"10cb022702c5754dbf2cd4dbfa44edd5d300ed6b9004227bbb72f6013c0095ef"}
Jul 17 18:17:48.837321 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:48.836972    2572 scope.go:117] "RemoveContainer" containerID="851888ed9bb55d1740998da755683819500425e7e53fa53f624705481eda3a12"
Jul 17 18:17:48.837321 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:48.837258    2572 scope.go:117] "RemoveContainer" containerID="10cb022702c5754dbf2cd4dbfa44edd5d300ed6b9004227bbb72f6013c0095ef"
Jul 17 18:17:48.837540 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:48.837515    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5_odh-ai-gateway-infra(43a56407-0d7f-49e2-bc70-676624abbea7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" podUID="43a56407-0d7f-49e2-bc70-676624abbea7"
Jul 17 18:17:48.856779 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:48.856746    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"]
Jul 17 18:17:49.983254 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:49.983223    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"
Jul 17 18:17:50.025767 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:50.025733    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a56407-0d7f-49e2-bc70-676624abbea7-maas-api-service-ca\") pod \"43a56407-0d7f-49e2-bc70-676624abbea7\" (UID: \"43a56407-0d7f-49e2-bc70-676624abbea7\") "
Jul 17 18:17:50.026128 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:50.026102    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43a56407-0d7f-49e2-bc70-676624abbea7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "43a56407-0d7f-49e2-bc70-676624abbea7" (UID: "43a56407-0d7f-49e2-bc70-676624abbea7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:17:50.126852 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:50.126752    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a56407-0d7f-49e2-bc70-676624abbea7-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:17:50.856825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:50.856785    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5" event={"ID":"43a56407-0d7f-49e2-bc70-676624abbea7","Type":"ContainerDied","Data":"b49708adf288a8291023c3af992903bc6be3e8dc21796db9b3dd7020f36f6b9e"}
Jul 17 18:17:50.856825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:50.856824    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"
Jul 17 18:17:50.857143 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:50.856870    2572 scope.go:117] "RemoveContainer" containerID="10cb022702c5754dbf2cd4dbfa44edd5d300ed6b9004227bbb72f6013c0095ef"
Jul 17 18:17:50.875636 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:50.875597    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"]
Jul 17 18:17:50.880834 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:50.880803    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-czfb5"]
Jul 17 18:17:52.516468 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:52.516425    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" path="/var/lib/kubelet/pods/43a56407-0d7f-49e2-bc70-676624abbea7/volumes"
Jul 17 18:17:54.814074 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814033    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"]
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814379    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814390    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814400    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814406    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814428    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814434    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814445    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814470    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814478    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814483    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814509    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49b8ba94-5cc8-4a8d-b7a4-a72aad654786" containerName="main"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814515    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="49b8ba94-5cc8-4a8d-b7a4-a72aad654786" containerName="main"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814521    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814526    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814533    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerName="revoke-keys"
Jul 17 18:17:54.814547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814541    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerName="revoke-keys"
Jul 17 18:17:54.815077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814598    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerName="revoke-keys"
Jul 17 18:17:54.815077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814604    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" containerName="revoke-keys"
Jul 17 18:17:54.815077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814609    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" containerName="revoke-keys"
Jul 17 18:17:54.815077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814615    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" containerName="revoke-keys"
Jul 17 18:17:54.815077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814621    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="49b8ba94-5cc8-4a8d-b7a4-a72aad654786" containerName="main"
Jul 17 18:17:54.815077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814627    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerName="revoke-keys"
Jul 17 18:17:54.815077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.814635    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" containerName="revoke-keys"
Jul 17 18:17:54.819237 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.819214    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"
Jul 17 18:17:54.822161 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.822131    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:17:54.822353 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.822290    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:17:54.825844 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.825816    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"]
Jul 17 18:17:54.870114 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.870065    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3338763-1113-433f-88db-3702611f4957-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7\" (UID: \"b3338763-1113-433f-88db-3702611f4957\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"
Jul 17 18:17:54.971247 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.971206    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3338763-1113-433f-88db-3702611f4957-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7\" (UID: \"b3338763-1113-433f-88db-3702611f4957\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"
Jul 17 18:17:54.971923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:54.971901    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3338763-1113-433f-88db-3702611f4957-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7\" (UID: \"b3338763-1113-433f-88db-3702611f4957\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"
Jul 17 18:17:55.133109 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:55.133018    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"
Jul 17 18:17:55.279392 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:55.279368    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"]
Jul 17 18:17:55.281320 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:17:55.281291    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3338763_1113_433f_88db_3702611f4957.slice/crio-82595f0541700fe2dd018a6ce58193037c3a65de0444a97b2646d5a20b83f9ac WatchSource:0}: Error finding container 82595f0541700fe2dd018a6ce58193037c3a65de0444a97b2646d5a20b83f9ac: Status 404 returned error can't find the container with id 82595f0541700fe2dd018a6ce58193037c3a65de0444a97b2646d5a20b83f9ac
Jul 17 18:17:55.890589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:55.890551    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" event={"ID":"b3338763-1113-433f-88db-3702611f4957","Type":"ContainerStarted","Data":"234f7f47341052f99a7382fee4951ab0bb98c2a32143ace517227c1443ce0b0e"}
Jul 17 18:17:55.891066 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:55.890614    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" event={"ID":"b3338763-1113-433f-88db-3702611f4957","Type":"ContainerStarted","Data":"82595f0541700fe2dd018a6ce58193037c3a65de0444a97b2646d5a20b83f9ac"}
Jul 17 18:17:55.906904 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:55.906846    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" podStartSLOduration=1.9068271879999998 podStartE2EDuration="1.906827188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:17:55.904988381 +0000 UTC m=+3247.910344236" watchObservedRunningTime="2026-07-17 18:17:55.906827188 +0000 UTC m=+3247.912183105"
Jul 17 18:17:56.897920 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:56.897880    2572 generic.go:356] "Generic (PLEG): container finished" podID="b3338763-1113-433f-88db-3702611f4957" containerID="234f7f47341052f99a7382fee4951ab0bb98c2a32143ace517227c1443ce0b0e" exitCode=6
Jul 17 18:17:56.898406 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:56.897978    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" event={"ID":"b3338763-1113-433f-88db-3702611f4957","Type":"ContainerDied","Data":"234f7f47341052f99a7382fee4951ab0bb98c2a32143ace517227c1443ce0b0e"}
Jul 17 18:17:56.898406 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:56.898177    2572 scope.go:117] "RemoveContainer" containerID="234f7f47341052f99a7382fee4951ab0bb98c2a32143ace517227c1443ce0b0e"
Jul 17 18:17:57.906777 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:57.906670    2572 generic.go:356] "Generic (PLEG): container finished" podID="b3338763-1113-433f-88db-3702611f4957" containerID="d2d5a9c4a5981d503365cc969eae27855f9ca49a50be86bedb7ae0dafdf04f27" exitCode=6
Jul 17 18:17:57.906777 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:57.906732    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" event={"ID":"b3338763-1113-433f-88db-3702611f4957","Type":"ContainerDied","Data":"d2d5a9c4a5981d503365cc969eae27855f9ca49a50be86bedb7ae0dafdf04f27"}
Jul 17 18:17:57.907310 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:57.906790    2572 scope.go:117] "RemoveContainer" containerID="234f7f47341052f99a7382fee4951ab0bb98c2a32143ace517227c1443ce0b0e"
Jul 17 18:17:57.907310 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:57.907056    2572 scope.go:117] "RemoveContainer" containerID="d2d5a9c4a5981d503365cc969eae27855f9ca49a50be86bedb7ae0dafdf04f27"
Jul 17 18:17:57.907414 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:57.907306    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7_odh-ai-gateway-infra(b3338763-1113-433f-88db-3702611f4957)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" podUID="b3338763-1113-433f-88db-3702611f4957"
Jul 17 18:17:58.913680 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:17:58.913644    2572 scope.go:117] "RemoveContainer" containerID="d2d5a9c4a5981d503365cc969eae27855f9ca49a50be86bedb7ae0dafdf04f27"
Jul 17 18:17:58.914164 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:17:58.913861    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7_odh-ai-gateway-infra(b3338763-1113-433f-88db-3702611f4957)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" podUID="b3338763-1113-433f-88db-3702611f4957"
Jul 17 18:18:10.510358 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.510315    2572 scope.go:117] "RemoveContainer" containerID="d2d5a9c4a5981d503365cc969eae27855f9ca49a50be86bedb7ae0dafdf04f27"
Jul 17 18:18:10.600626 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.600587    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"]
Jul 17 18:18:10.600953 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.600942    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" containerName="revoke-keys"
Jul 17 18:18:10.600993 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.600955    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" containerName="revoke-keys"
Jul 17 18:18:10.600993 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.600967    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerName="revoke-keys"
Jul 17 18:18:10.600993 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.600972    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerName="revoke-keys"
Jul 17 18:18:10.601086 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.601033    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5170e57-9eca-4bcd-9a6f-7108da612974" containerName="revoke-keys"
Jul 17 18:18:10.601086 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.601042    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="43a56407-0d7f-49e2-bc70-676624abbea7" containerName="revoke-keys"
Jul 17 18:18:10.601086 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.601048    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f157795d-0b0f-4083-b723-18f8afb831f5" containerName="revoke-keys"
Jul 17 18:18:10.605057 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.605040    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"
Jul 17 18:18:10.611682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.611627    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"]
Jul 17 18:18:10.729857 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.729827    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d736e021-8d90-49ae-a574-4affab671306-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt\" (UID: \"d736e021-8d90-49ae-a574-4affab671306\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"
Jul 17 18:18:10.830987 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.830944    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d736e021-8d90-49ae-a574-4affab671306-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt\" (UID: \"d736e021-8d90-49ae-a574-4affab671306\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"
Jul 17 18:18:10.831600 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.831582    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d736e021-8d90-49ae-a574-4affab671306-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt\" (UID: \"d736e021-8d90-49ae-a574-4affab671306\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"
Jul 17 18:18:10.917923 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.917885    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"
Jul 17 18:18:10.988902 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:10.988863    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" event={"ID":"b3338763-1113-433f-88db-3702611f4957","Type":"ContainerStarted","Data":"b582d36a3ab496de5f3d69d22b8db6b4e3c3b01d5c9323328cb5e6c2a4787a38"}
Jul 17 18:18:11.068508 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:11.068455    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"]
Jul 17 18:18:11.069938 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:18:11.069909    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd736e021_8d90_49ae_a574_4affab671306.slice/crio-99a422230c5f86031b759032d662cb3f0fd9683df575980764c2d35c54f01e4a WatchSource:0}: Error finding container 99a422230c5f86031b759032d662cb3f0fd9683df575980764c2d35c54f01e4a: Status 404 returned error can't find the container with id 99a422230c5f86031b759032d662cb3f0fd9683df575980764c2d35c54f01e4a
Jul 17 18:18:11.998543 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:11.998508    2572 generic.go:356] "Generic (PLEG): container finished" podID="b3338763-1113-433f-88db-3702611f4957" containerID="b582d36a3ab496de5f3d69d22b8db6b4e3c3b01d5c9323328cb5e6c2a4787a38" exitCode=6
Jul 17 18:18:11.998974 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:11.998570    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" event={"ID":"b3338763-1113-433f-88db-3702611f4957","Type":"ContainerDied","Data":"b582d36a3ab496de5f3d69d22b8db6b4e3c3b01d5c9323328cb5e6c2a4787a38"}
Jul 17 18:18:11.998974 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:11.998669    2572 scope.go:117] "RemoveContainer" containerID="d2d5a9c4a5981d503365cc969eae27855f9ca49a50be86bedb7ae0dafdf04f27"
Jul 17 18:18:11.999096 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:11.998981    2572 scope.go:117] "RemoveContainer" containerID="b582d36a3ab496de5f3d69d22b8db6b4e3c3b01d5c9323328cb5e6c2a4787a38"
Jul 17 18:18:11.999255 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:11.999229    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7_odh-ai-gateway-infra(b3338763-1113-433f-88db-3702611f4957)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" podUID="b3338763-1113-433f-88db-3702611f4957"
Jul 17 18:18:12.000424 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:12.000401    2572 generic.go:356] "Generic (PLEG): container finished" podID="d736e021-8d90-49ae-a574-4affab671306" containerID="56faf68d0a302e05a403218a6bddc2f66bebe68394dec765ec7c520216e8db69" exitCode=6
Jul 17 18:18:12.000532 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:12.000484    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" event={"ID":"d736e021-8d90-49ae-a574-4affab671306","Type":"ContainerDied","Data":"56faf68d0a302e05a403218a6bddc2f66bebe68394dec765ec7c520216e8db69"}
Jul 17 18:18:12.000584 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:12.000553    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" event={"ID":"d736e021-8d90-49ae-a574-4affab671306","Type":"ContainerStarted","Data":"99a422230c5f86031b759032d662cb3f0fd9683df575980764c2d35c54f01e4a"}
Jul 17 18:18:12.000660 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:12.000648    2572 scope.go:117] "RemoveContainer" containerID="56faf68d0a302e05a403218a6bddc2f66bebe68394dec765ec7c520216e8db69"
Jul 17 18:18:12.020233 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:12.020204    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"]
Jul 17 18:18:13.011362 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:13.011329    2572 generic.go:356] "Generic (PLEG): container finished" podID="d736e021-8d90-49ae-a574-4affab671306" containerID="95d38862e2283ba9e6409fbaaeae3936aeb8c48eca1a93185984887fb7ac4041" exitCode=6
Jul 17 18:18:13.011836 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:13.011418    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" event={"ID":"d736e021-8d90-49ae-a574-4affab671306","Type":"ContainerDied","Data":"95d38862e2283ba9e6409fbaaeae3936aeb8c48eca1a93185984887fb7ac4041"}
Jul 17 18:18:13.011836 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:13.011509    2572 scope.go:117] "RemoveContainer" containerID="56faf68d0a302e05a403218a6bddc2f66bebe68394dec765ec7c520216e8db69"
Jul 17 18:18:13.011836 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:13.011803    2572 scope.go:117] "RemoveContainer" containerID="95d38862e2283ba9e6409fbaaeae3936aeb8c48eca1a93185984887fb7ac4041"
Jul 17 18:18:13.012101 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:13.012076    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt_odh-ai-gateway-infra(d736e021-8d90-49ae-a574-4affab671306)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" podUID="d736e021-8d90-49ae-a574-4affab671306"
Jul 17 18:18:13.150658 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:13.150632    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"
Jul 17 18:18:13.252315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:13.252280    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3338763-1113-433f-88db-3702611f4957-maas-api-service-ca\") pod \"b3338763-1113-433f-88db-3702611f4957\" (UID: \"b3338763-1113-433f-88db-3702611f4957\") "
Jul 17 18:18:13.252732 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:13.252708    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3338763-1113-433f-88db-3702611f4957-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3338763-1113-433f-88db-3702611f4957" (UID: "b3338763-1113-433f-88db-3702611f4957"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:18:13.353332 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:13.353296    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3338763-1113-433f-88db-3702611f4957-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:18:14.018248 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:14.018213    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"
Jul 17 18:18:14.018771 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:14.018247    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7" event={"ID":"b3338763-1113-433f-88db-3702611f4957","Type":"ContainerDied","Data":"82595f0541700fe2dd018a6ce58193037c3a65de0444a97b2646d5a20b83f9ac"}
Jul 17 18:18:14.018771 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:14.018319    2572 scope.go:117] "RemoveContainer" containerID="b582d36a3ab496de5f3d69d22b8db6b4e3c3b01d5c9323328cb5e6c2a4787a38"
Jul 17 18:18:14.020174 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:14.020157    2572 scope.go:117] "RemoveContainer" containerID="95d38862e2283ba9e6409fbaaeae3936aeb8c48eca1a93185984887fb7ac4041"
Jul 17 18:18:14.020369 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:14.020351    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt_odh-ai-gateway-infra(d736e021-8d90-49ae-a574-4affab671306)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" podUID="d736e021-8d90-49ae-a574-4affab671306"
Jul 17 18:18:14.059513 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:14.059455    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"]
Jul 17 18:18:14.063935 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:14.063908    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-nlpc7"]
Jul 17 18:18:14.516231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:14.516194    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3338763-1113-433f-88db-3702611f4957" path="/var/lib/kubelet/pods/b3338763-1113-433f-88db-3702611f4957/volumes"
Jul 17 18:18:17.059795 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.059762    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"]
Jul 17 18:18:17.060201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.060092    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3338763-1113-433f-88db-3702611f4957" containerName="revoke-keys"
Jul 17 18:18:17.060201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.060102    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3338763-1113-433f-88db-3702611f4957" containerName="revoke-keys"
Jul 17 18:18:17.060201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.060110    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3338763-1113-433f-88db-3702611f4957" containerName="revoke-keys"
Jul 17 18:18:17.060201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.060115    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3338763-1113-433f-88db-3702611f4957" containerName="revoke-keys"
Jul 17 18:18:17.060201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.060190    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3338763-1113-433f-88db-3702611f4957" containerName="revoke-keys"
Jul 17 18:18:17.060201 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.060196    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3338763-1113-433f-88db-3702611f4957" containerName="revoke-keys"
Jul 17 18:18:17.064795 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.064774    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"
Jul 17 18:18:17.072243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.072214    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"]
Jul 17 18:18:17.189356 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.189316    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e68d33-00d0-4536-9e15-6d3fd6437df4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4\" (UID: \"e2e68d33-00d0-4536-9e15-6d3fd6437df4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"
Jul 17 18:18:17.289845 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.289811    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e68d33-00d0-4536-9e15-6d3fd6437df4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4\" (UID: \"e2e68d33-00d0-4536-9e15-6d3fd6437df4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"
Jul 17 18:18:17.290437 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.290407    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e68d33-00d0-4536-9e15-6d3fd6437df4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4\" (UID: \"e2e68d33-00d0-4536-9e15-6d3fd6437df4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"
Jul 17 18:18:17.376891 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.376788    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"
Jul 17 18:18:17.524827 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:17.524793    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"]
Jul 17 18:18:17.526769 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:18:17.526733    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e68d33_00d0_4536_9e15_6d3fd6437df4.slice/crio-f22566ff050869ef0961cc2e4612a8b65445b045244d7d67dadf18a9d24f0417 WatchSource:0}: Error finding container f22566ff050869ef0961cc2e4612a8b65445b045244d7d67dadf18a9d24f0417: Status 404 returned error can't find the container with id f22566ff050869ef0961cc2e4612a8b65445b045244d7d67dadf18a9d24f0417
Jul 17 18:18:18.045721 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:18.045673    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" event={"ID":"e2e68d33-00d0-4536-9e15-6d3fd6437df4","Type":"ContainerStarted","Data":"c21f6de15fa6b68fa24546cee9bdcdbb4bc4ca60e9fd2b9b0e268792507627d1"}
Jul 17 18:18:18.045922 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:18.045751    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" event={"ID":"e2e68d33-00d0-4536-9e15-6d3fd6437df4","Type":"ContainerStarted","Data":"f22566ff050869ef0961cc2e4612a8b65445b045244d7d67dadf18a9d24f0417"}
Jul 17 18:18:18.066009 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:18.065954    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" podStartSLOduration=1.065936526 podStartE2EDuration="1.065936526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:18:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:18:18.064359374 +0000 UTC m=+3270.069715241" watchObservedRunningTime="2026-07-17 18:18:18.065936526 +0000 UTC m=+3270.071292379"
Jul 17 18:18:19.053376 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:19.053338    2572 generic.go:356] "Generic (PLEG): container finished" podID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerID="c21f6de15fa6b68fa24546cee9bdcdbb4bc4ca60e9fd2b9b0e268792507627d1" exitCode=6
Jul 17 18:18:19.053596 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:19.053425    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" event={"ID":"e2e68d33-00d0-4536-9e15-6d3fd6437df4","Type":"ContainerDied","Data":"c21f6de15fa6b68fa24546cee9bdcdbb4bc4ca60e9fd2b9b0e268792507627d1"}
Jul 17 18:18:19.053728 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:19.053714    2572 scope.go:117] "RemoveContainer" containerID="c21f6de15fa6b68fa24546cee9bdcdbb4bc4ca60e9fd2b9b0e268792507627d1"
Jul 17 18:18:20.062058 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:20.062025    2572 generic.go:356] "Generic (PLEG): container finished" podID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerID="8d8e6238b5819715783de3fee1217d48c4d4bf72048aeb5c3574a3a49f3f9aef" exitCode=6
Jul 17 18:18:20.062606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:20.062102    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" event={"ID":"e2e68d33-00d0-4536-9e15-6d3fd6437df4","Type":"ContainerDied","Data":"8d8e6238b5819715783de3fee1217d48c4d4bf72048aeb5c3574a3a49f3f9aef"}
Jul 17 18:18:20.062606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:20.062166    2572 scope.go:117] "RemoveContainer" containerID="c21f6de15fa6b68fa24546cee9bdcdbb4bc4ca60e9fd2b9b0e268792507627d1"
Jul 17 18:18:20.062606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:20.062393    2572 scope.go:117] "RemoveContainer" containerID="8d8e6238b5819715783de3fee1217d48c4d4bf72048aeb5c3574a3a49f3f9aef"
Jul 17 18:18:20.062734 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:20.062706    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4_odh-ai-gateway-infra(e2e68d33-00d0-4536-9e15-6d3fd6437df4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4"
Jul 17 18:18:21.069644 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:21.069609    2572 scope.go:117] "RemoveContainer" containerID="8d8e6238b5819715783de3fee1217d48c4d4bf72048aeb5c3574a3a49f3f9aef"
Jul 17 18:18:21.070048 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:21.069837    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4_odh-ai-gateway-infra(e2e68d33-00d0-4536-9e15-6d3fd6437df4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4"
Jul 17 18:18:28.513049 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:28.513017    2572 scope.go:117] "RemoveContainer" containerID="95d38862e2283ba9e6409fbaaeae3936aeb8c48eca1a93185984887fb7ac4041"
Jul 17 18:18:28.966394 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:28.966356    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"]
Jul 17 18:18:28.966767 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:28.966753    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3338763-1113-433f-88db-3702611f4957" containerName="revoke-keys"
Jul 17 18:18:28.966814 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:28.966768    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3338763-1113-433f-88db-3702611f4957" containerName="revoke-keys"
Jul 17 18:18:28.966851 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:28.966837    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3338763-1113-433f-88db-3702611f4957" containerName="revoke-keys"
Jul 17 18:18:28.969931 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:28.969909    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"
Jul 17 18:18:28.977600 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:28.977522    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"]
Jul 17 18:18:28.981415 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:28.981382    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1dd56c-624d-470f-af39-08993a4c9673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk\" (UID: \"dc1dd56c-624d-470f-af39-08993a4c9673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"
Jul 17 18:18:29.082547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:29.082417    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1dd56c-624d-470f-af39-08993a4c9673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk\" (UID: \"dc1dd56c-624d-470f-af39-08993a4c9673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"
Jul 17 18:18:29.083082 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:29.083061    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1dd56c-624d-470f-af39-08993a4c9673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk\" (UID: \"dc1dd56c-624d-470f-af39-08993a4c9673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"
Jul 17 18:18:29.118531 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:29.118476    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" event={"ID":"d736e021-8d90-49ae-a574-4affab671306","Type":"ContainerStarted","Data":"adf2984852c2d958da3dd7d4914fa43881a23ef05ce2acc1e92c2321c040b731"}
Jul 17 18:18:29.138435 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:29.138361    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" podStartSLOduration=19.138336535 podStartE2EDuration="19.138336535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:18:29.133726797 +0000 UTC m=+3281.139082652" watchObservedRunningTime="2026-07-17 18:18:29.138336535 +0000 UTC m=+3281.143692392"
Jul 17 18:18:29.282811 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:29.282772    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"
Jul 17 18:18:29.433115 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:29.433089    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"]
Jul 17 18:18:29.434854 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:18:29.434826    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1dd56c_624d_470f_af39_08993a4c9673.slice/crio-24bca5f7836dc60a484ba6c415ce8d72b5ba322f746e3acb5301f83f7f062eed WatchSource:0}: Error finding container 24bca5f7836dc60a484ba6c415ce8d72b5ba322f746e3acb5301f83f7f062eed: Status 404 returned error can't find the container with id 24bca5f7836dc60a484ba6c415ce8d72b5ba322f746e3acb5301f83f7f062eed
Jul 17 18:18:30.127738 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:30.127696    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" event={"ID":"dc1dd56c-624d-470f-af39-08993a4c9673","Type":"ContainerStarted","Data":"9a06907893d4cd899a389174f3b08e703cb5374e28cb9ac2a4756c65bcfbdb38"}
Jul 17 18:18:30.128192 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:30.127767    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" event={"ID":"dc1dd56c-624d-470f-af39-08993a4c9673","Type":"ContainerStarted","Data":"24bca5f7836dc60a484ba6c415ce8d72b5ba322f746e3acb5301f83f7f062eed"}
Jul 17 18:18:30.129864 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:30.129834    2572 generic.go:356] "Generic (PLEG): container finished" podID="d736e021-8d90-49ae-a574-4affab671306" containerID="adf2984852c2d958da3dd7d4914fa43881a23ef05ce2acc1e92c2321c040b731" exitCode=6
Jul 17 18:18:30.130024 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:30.129904    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" event={"ID":"d736e021-8d90-49ae-a574-4affab671306","Type":"ContainerDied","Data":"adf2984852c2d958da3dd7d4914fa43881a23ef05ce2acc1e92c2321c040b731"}
Jul 17 18:18:30.130024 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:30.129960    2572 scope.go:117] "RemoveContainer" containerID="95d38862e2283ba9e6409fbaaeae3936aeb8c48eca1a93185984887fb7ac4041"
Jul 17 18:18:30.130295 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:30.130276    2572 scope.go:117] "RemoveContainer" containerID="adf2984852c2d958da3dd7d4914fa43881a23ef05ce2acc1e92c2321c040b731"
Jul 17 18:18:30.130608 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:30.130581    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt_odh-ai-gateway-infra(d736e021-8d90-49ae-a574-4affab671306)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" podUID="d736e021-8d90-49ae-a574-4affab671306"
Jul 17 18:18:30.143825 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:30.143772    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" podStartSLOduration=2.143754204 podStartE2EDuration="2.143754204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:18:30.142825956 +0000 UTC m=+3282.148181810" watchObservedRunningTime="2026-07-17 18:18:30.143754204 +0000 UTC m=+3282.149110113"
Jul 17 18:18:30.148642 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:30.148595    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"]
Jul 17 18:18:31.138179 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:31.138140    2572 generic.go:356] "Generic (PLEG): container finished" podID="dc1dd56c-624d-470f-af39-08993a4c9673" containerID="9a06907893d4cd899a389174f3b08e703cb5374e28cb9ac2a4756c65bcfbdb38" exitCode=6
Jul 17 18:18:31.138741 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:31.138239    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" event={"ID":"dc1dd56c-624d-470f-af39-08993a4c9673","Type":"ContainerDied","Data":"9a06907893d4cd899a389174f3b08e703cb5374e28cb9ac2a4756c65bcfbdb38"}
Jul 17 18:18:31.138741 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:31.138447    2572 scope.go:117] "RemoveContainer" containerID="9a06907893d4cd899a389174f3b08e703cb5374e28cb9ac2a4756c65bcfbdb38"
Jul 17 18:18:31.286303 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:31.286275    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"
Jul 17 18:18:31.299759 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:31.299712    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d736e021-8d90-49ae-a574-4affab671306-maas-api-service-ca\") pod \"d736e021-8d90-49ae-a574-4affab671306\" (UID: \"d736e021-8d90-49ae-a574-4affab671306\") "
Jul 17 18:18:31.300078 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:31.300056    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d736e021-8d90-49ae-a574-4affab671306-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d736e021-8d90-49ae-a574-4affab671306" (UID: "d736e021-8d90-49ae-a574-4affab671306"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:18:31.401032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:31.400939    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d736e021-8d90-49ae-a574-4affab671306-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:18:32.147621 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.147514    2572 generic.go:356] "Generic (PLEG): container finished" podID="dc1dd56c-624d-470f-af39-08993a4c9673" containerID="415690dc4d38e8ea9224cea0713a40428eb8bbd8b4c7ea9595e6dffb8276ad1b" exitCode=6
Jul 17 18:18:32.147621 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.147571    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" event={"ID":"dc1dd56c-624d-470f-af39-08993a4c9673","Type":"ContainerDied","Data":"415690dc4d38e8ea9224cea0713a40428eb8bbd8b4c7ea9595e6dffb8276ad1b"}
Jul 17 18:18:32.148184 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.147643    2572 scope.go:117] "RemoveContainer" containerID="9a06907893d4cd899a389174f3b08e703cb5374e28cb9ac2a4756c65bcfbdb38"
Jul 17 18:18:32.148184 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.147868    2572 scope.go:117] "RemoveContainer" containerID="415690dc4d38e8ea9224cea0713a40428eb8bbd8b4c7ea9595e6dffb8276ad1b"
Jul 17 18:18:32.148184 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:32.148107    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk_odh-ai-gateway-infra(dc1dd56c-624d-470f-af39-08993a4c9673)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" podUID="dc1dd56c-624d-470f-af39-08993a4c9673"
Jul 17 18:18:32.149087 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.149070    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"
Jul 17 18:18:32.149162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.149093    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt" event={"ID":"d736e021-8d90-49ae-a574-4affab671306","Type":"ContainerDied","Data":"99a422230c5f86031b759032d662cb3f0fd9683df575980764c2d35c54f01e4a"}
Jul 17 18:18:32.160027 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.160008    2572 scope.go:117] "RemoveContainer" containerID="adf2984852c2d958da3dd7d4914fa43881a23ef05ce2acc1e92c2321c040b731"
Jul 17 18:18:32.178234 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.178199    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"]
Jul 17 18:18:32.184218 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.184182    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-xsfxt"]
Jul 17 18:18:32.517305 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:32.517264    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d736e021-8d90-49ae-a574-4affab671306" path="/var/lib/kubelet/pods/d736e021-8d90-49ae-a574-4affab671306/volumes"
Jul 17 18:18:33.157163 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:33.157134    2572 scope.go:117] "RemoveContainer" containerID="415690dc4d38e8ea9224cea0713a40428eb8bbd8b4c7ea9595e6dffb8276ad1b"
Jul 17 18:18:33.157650 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:33.157427    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk_odh-ai-gateway-infra(dc1dd56c-624d-470f-af39-08993a4c9673)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" podUID="dc1dd56c-624d-470f-af39-08993a4c9673"
Jul 17 18:18:35.510650 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:35.510613    2572 scope.go:117] "RemoveContainer" containerID="8d8e6238b5819715783de3fee1217d48c4d4bf72048aeb5c3574a3a49f3f9aef"
Jul 17 18:18:36.179994 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:36.179954    2572 generic.go:356] "Generic (PLEG): container finished" podID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerID="216ae28baa8bf505b2120c1d4b4cdb66f0be2ef0e4e38de60f9e5b71e55a21dd" exitCode=6
Jul 17 18:18:36.180254 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:36.180018    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" event={"ID":"e2e68d33-00d0-4536-9e15-6d3fd6437df4","Type":"ContainerDied","Data":"216ae28baa8bf505b2120c1d4b4cdb66f0be2ef0e4e38de60f9e5b71e55a21dd"}
Jul 17 18:18:36.180254 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:36.180083    2572 scope.go:117] "RemoveContainer" containerID="8d8e6238b5819715783de3fee1217d48c4d4bf72048aeb5c3574a3a49f3f9aef"
Jul 17 18:18:36.180417 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:36.180400    2572 scope.go:117] "RemoveContainer" containerID="216ae28baa8bf505b2120c1d4b4cdb66f0be2ef0e4e38de60f9e5b71e55a21dd"
Jul 17 18:18:36.180685 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:36.180663    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4_odh-ai-gateway-infra(e2e68d33-00d0-4536-9e15-6d3fd6437df4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4"
Jul 17 18:18:37.207568 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:37.207529    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"]
Jul 17 18:18:37.352726 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:37.352691    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"
Jul 17 18:18:37.442901 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:37.442850    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e68d33-00d0-4536-9e15-6d3fd6437df4-maas-api-service-ca\") pod \"e2e68d33-00d0-4536-9e15-6d3fd6437df4\" (UID: \"e2e68d33-00d0-4536-9e15-6d3fd6437df4\") "
Jul 17 18:18:37.443282 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:37.443253    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2e68d33-00d0-4536-9e15-6d3fd6437df4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2e68d33-00d0-4536-9e15-6d3fd6437df4" (UID: "e2e68d33-00d0-4536-9e15-6d3fd6437df4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:18:37.543996 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:37.543882    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e68d33-00d0-4536-9e15-6d3fd6437df4-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:18:38.194873 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:38.194846    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"
Jul 17 18:18:38.195049 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:38.194878    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4" event={"ID":"e2e68d33-00d0-4536-9e15-6d3fd6437df4","Type":"ContainerDied","Data":"f22566ff050869ef0961cc2e4612a8b65445b045244d7d67dadf18a9d24f0417"}
Jul 17 18:18:38.195049 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:38.194946    2572 scope.go:117] "RemoveContainer" containerID="216ae28baa8bf505b2120c1d4b4cdb66f0be2ef0e4e38de60f9e5b71e55a21dd"
Jul 17 18:18:38.220219 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:38.220181    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"]
Jul 17 18:18:38.225877 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:38.225844    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-dxwx4"]
Jul 17 18:18:38.515929 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:38.515844    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" path="/var/lib/kubelet/pods/e2e68d33-00d0-4536-9e15-6d3fd6437df4/volumes"
Jul 17 18:18:44.861527 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861473    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"]
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861822    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d736e021-8d90-49ae-a574-4affab671306" containerName="revoke-keys"
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861832    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d736e021-8d90-49ae-a574-4affab671306" containerName="revoke-keys"
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861847    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerName="revoke-keys"
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861853    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerName="revoke-keys"
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861862    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerName="revoke-keys"
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861867    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerName="revoke-keys"
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861879    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerName="revoke-keys"
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861884    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerName="revoke-keys"
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861892    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d736e021-8d90-49ae-a574-4affab671306" containerName="revoke-keys"
Jul 17 18:18:44.861934 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861899    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d736e021-8d90-49ae-a574-4affab671306" containerName="revoke-keys"
Jul 17 18:18:44.862243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861956    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerName="revoke-keys"
Jul 17 18:18:44.862243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861963    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerName="revoke-keys"
Jul 17 18:18:44.862243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861970    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2e68d33-00d0-4536-9e15-6d3fd6437df4" containerName="revoke-keys"
Jul 17 18:18:44.862243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861978    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d736e021-8d90-49ae-a574-4affab671306" containerName="revoke-keys"
Jul 17 18:18:44.862243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.861984    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d736e021-8d90-49ae-a574-4affab671306" containerName="revoke-keys"
Jul 17 18:18:44.866522 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.866480    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"
Jul 17 18:18:44.874132 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.874095    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"]
Jul 17 18:18:44.896386 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.896342    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9593bf8f-7084-4512-a714-be41ed03c85d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc\" (UID: \"9593bf8f-7084-4512-a714-be41ed03c85d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"
Jul 17 18:18:44.997458 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.997412    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9593bf8f-7084-4512-a714-be41ed03c85d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc\" (UID: \"9593bf8f-7084-4512-a714-be41ed03c85d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"
Jul 17 18:18:44.998133 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:44.998106    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9593bf8f-7084-4512-a714-be41ed03c85d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc\" (UID: \"9593bf8f-7084-4512-a714-be41ed03c85d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"
Jul 17 18:18:45.178879 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:45.178775    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"
Jul 17 18:18:45.326082 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:45.326041    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"]
Jul 17 18:18:45.326604 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:18:45.326565    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9593bf8f_7084_4512_a714_be41ed03c85d.slice/crio-bfada8f3f7e2e3352e0304473b22ba7edfa80ae91f13993a40bbc9db47445424 WatchSource:0}: Error finding container bfada8f3f7e2e3352e0304473b22ba7edfa80ae91f13993a40bbc9db47445424: Status 404 returned error can't find the container with id bfada8f3f7e2e3352e0304473b22ba7edfa80ae91f13993a40bbc9db47445424
Jul 17 18:18:45.510456 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:45.510361    2572 scope.go:117] "RemoveContainer" containerID="415690dc4d38e8ea9224cea0713a40428eb8bbd8b4c7ea9595e6dffb8276ad1b"
Jul 17 18:18:46.249393 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:46.249354    2572 generic.go:356] "Generic (PLEG): container finished" podID="9593bf8f-7084-4512-a714-be41ed03c85d" containerID="712145a845d6d9770b827fb454f9f6d31a76276c3804d7a1fdc65b7eceadc38a" exitCode=6
Jul 17 18:18:46.249870 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:46.249430    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc" event={"ID":"9593bf8f-7084-4512-a714-be41ed03c85d","Type":"ContainerDied","Data":"712145a845d6d9770b827fb454f9f6d31a76276c3804d7a1fdc65b7eceadc38a"}
Jul 17 18:18:46.249870 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:46.249482    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc" event={"ID":"9593bf8f-7084-4512-a714-be41ed03c85d","Type":"ContainerStarted","Data":"bfada8f3f7e2e3352e0304473b22ba7edfa80ae91f13993a40bbc9db47445424"}
Jul 17 18:18:46.249870 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:46.249669    2572 scope.go:117] "RemoveContainer" containerID="712145a845d6d9770b827fb454f9f6d31a76276c3804d7a1fdc65b7eceadc38a"
Jul 17 18:18:46.251617 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:46.251593    2572 generic.go:356] "Generic (PLEG): container finished" podID="dc1dd56c-624d-470f-af39-08993a4c9673" containerID="5eb26d3810076bfa55cd48a32f35bbb91f8a01b1102358d75138efecfa7a20bd" exitCode=6
Jul 17 18:18:46.251727 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:46.251630    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" event={"ID":"dc1dd56c-624d-470f-af39-08993a4c9673","Type":"ContainerDied","Data":"5eb26d3810076bfa55cd48a32f35bbb91f8a01b1102358d75138efecfa7a20bd"}
Jul 17 18:18:46.251727 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:46.251682    2572 scope.go:117] "RemoveContainer" containerID="415690dc4d38e8ea9224cea0713a40428eb8bbd8b4c7ea9595e6dffb8276ad1b"
Jul 17 18:18:46.252110 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:46.252093    2572 scope.go:117] "RemoveContainer" containerID="5eb26d3810076bfa55cd48a32f35bbb91f8a01b1102358d75138efecfa7a20bd"
Jul 17 18:18:46.252328 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:46.252306    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk_odh-ai-gateway-infra(dc1dd56c-624d-470f-af39-08993a4c9673)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" podUID="dc1dd56c-624d-470f-af39-08993a4c9673"
Jul 17 18:18:47.259952 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:47.259843    2572 generic.go:356] "Generic (PLEG): container finished" podID="9593bf8f-7084-4512-a714-be41ed03c85d" containerID="1102470603df9b5b867ea4cd0e48389ed68323c535c3a44e18c0a6081956c5f6" exitCode=6
Jul 17 18:18:47.259952 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:47.259939    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc" event={"ID":"9593bf8f-7084-4512-a714-be41ed03c85d","Type":"ContainerDied","Data":"1102470603df9b5b867ea4cd0e48389ed68323c535c3a44e18c0a6081956c5f6"}
Jul 17 18:18:47.260543 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:47.260005    2572 scope.go:117] "RemoveContainer" containerID="712145a845d6d9770b827fb454f9f6d31a76276c3804d7a1fdc65b7eceadc38a"
Jul 17 18:18:47.260543 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:47.260224    2572 scope.go:117] "RemoveContainer" containerID="1102470603df9b5b867ea4cd0e48389ed68323c535c3a44e18c0a6081956c5f6"
Jul 17 18:18:47.260543 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:47.260443    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc_odh-ai-gateway-infra(9593bf8f-7084-4512-a714-be41ed03c85d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc" podUID="9593bf8f-7084-4512-a714-be41ed03c85d"
Jul 17 18:18:47.289061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:47.289024    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"]
Jul 17 18:18:47.440035 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:47.440006    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"
Jul 17 18:18:47.518127 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:47.518011    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1dd56c-624d-470f-af39-08993a4c9673-maas-api-service-ca\") pod \"dc1dd56c-624d-470f-af39-08993a4c9673\" (UID: \"dc1dd56c-624d-470f-af39-08993a4c9673\") "
Jul 17 18:18:47.518475 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:47.518449    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc1dd56c-624d-470f-af39-08993a4c9673-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc1dd56c-624d-470f-af39-08993a4c9673" (UID: "dc1dd56c-624d-470f-af39-08993a4c9673"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:18:47.619211 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:47.619167    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1dd56c-624d-470f-af39-08993a4c9673-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:18:48.269919 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.269885    2572 scope.go:117] "RemoveContainer" containerID="1102470603df9b5b867ea4cd0e48389ed68323c535c3a44e18c0a6081956c5f6"
Jul 17 18:18:48.270383 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:18:48.270105    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc_odh-ai-gateway-infra(9593bf8f-7084-4512-a714-be41ed03c85d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc" podUID="9593bf8f-7084-4512-a714-be41ed03c85d"
Jul 17 18:18:48.271140 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.271113    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk" event={"ID":"dc1dd56c-624d-470f-af39-08993a4c9673","Type":"ContainerDied","Data":"24bca5f7836dc60a484ba6c415ce8d72b5ba322f746e3acb5301f83f7f062eed"}
Jul 17 18:18:48.271278 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.271142    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"
Jul 17 18:18:48.271278 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.271172    2572 scope.go:117] "RemoveContainer" containerID="5eb26d3810076bfa55cd48a32f35bbb91f8a01b1102358d75138efecfa7a20bd"
Jul 17 18:18:48.300610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.300557    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"]
Jul 17 18:18:48.306508 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.306463    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-szxbk"]
Jul 17 18:18:48.515369 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.515335    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" path="/var/lib/kubelet/pods/dc1dd56c-624d-470f-af39-08993a4c9673/volumes"
Jul 17 18:18:48.769918 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.769892    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 18:18:48.776652 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.776628    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 18:18:48.781808 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.781781    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 18:18:48.787738 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:48.787715    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bbmxm_00473330-68a9-4b11-8a60-732329b1891a/ovn-acl-logging/0.log"
Jul 17 18:18:59.510330 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:18:59.510289    2572 scope.go:117] "RemoveContainer" containerID="1102470603df9b5b867ea4cd0e48389ed68323c535c3a44e18c0a6081956c5f6"
Jul 17 18:19:00.342719 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:00.342685    2572 generic.go:356] "Generic (PLEG): container finished" podID="9593bf8f-7084-4512-a714-be41ed03c85d" containerID="a7b0337fbe558d2f767deeee010b759b551599a42f780e6b5d949ae0a03c5897" exitCode=6
Jul 17 18:19:00.342936 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:00.342764    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc" event={"ID":"9593bf8f-7084-4512-a714-be41ed03c85d","Type":"ContainerDied","Data":"a7b0337fbe558d2f767deeee010b759b551599a42f780e6b5d949ae0a03c5897"}
Jul 17 18:19:00.342936 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:00.342842    2572 scope.go:117] "RemoveContainer" containerID="1102470603df9b5b867ea4cd0e48389ed68323c535c3a44e18c0a6081956c5f6"
Jul 17 18:19:00.343163 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:00.343143    2572 scope.go:117] "RemoveContainer" containerID="a7b0337fbe558d2f767deeee010b759b551599a42f780e6b5d949ae0a03c5897"
Jul 17 18:19:00.343407 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:00.343382    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc_odh-ai-gateway-infra(9593bf8f-7084-4512-a714-be41ed03c85d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc" podUID="9593bf8f-7084-4512-a714-be41ed03c85d"
Jul 17 18:19:01.368846 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:01.368807    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"]
Jul 17 18:19:01.510379 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:01.510351    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"
Jul 17 18:19:01.546233 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:01.546200    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9593bf8f-7084-4512-a714-be41ed03c85d-maas-api-service-ca\") pod \"9593bf8f-7084-4512-a714-be41ed03c85d\" (UID: \"9593bf8f-7084-4512-a714-be41ed03c85d\") "
Jul 17 18:19:01.546711 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:01.546666    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9593bf8f-7084-4512-a714-be41ed03c85d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9593bf8f-7084-4512-a714-be41ed03c85d" (UID: "9593bf8f-7084-4512-a714-be41ed03c85d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:19:01.647807 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:01.647704    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9593bf8f-7084-4512-a714-be41ed03c85d-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:19:02.357445 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:02.357408    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"
Jul 17 18:19:02.357700 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:02.357441    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc" event={"ID":"9593bf8f-7084-4512-a714-be41ed03c85d","Type":"ContainerDied","Data":"bfada8f3f7e2e3352e0304473b22ba7edfa80ae91f13993a40bbc9db47445424"}
Jul 17 18:19:02.357700 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:02.357540    2572 scope.go:117] "RemoveContainer" containerID="a7b0337fbe558d2f767deeee010b759b551599a42f780e6b5d949ae0a03c5897"
Jul 17 18:19:02.381739 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:02.381702    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"]
Jul 17 18:19:02.387606 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:02.387574    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-bdrdc"]
Jul 17 18:19:02.515456 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:02.515417    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" path="/var/lib/kubelet/pods/9593bf8f-7084-4512-a714-be41ed03c85d/volumes"
Jul 17 18:19:10.644356 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644266    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"]
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644627    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644640    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644653    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644659    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644664    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644670    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644683    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d736e021-8d90-49ae-a574-4affab671306" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644689    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d736e021-8d90-49ae-a574-4affab671306" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644698    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644703    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644757    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644765    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644772    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d736e021-8d90-49ae-a574-4affab671306" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644780    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" containerName="revoke-keys"
Jul 17 18:19:10.644791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.644785    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" containerName="revoke-keys"
Jul 17 18:19:10.649327 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.649306    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"
Jul 17 18:19:10.653838 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.653627    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:19:10.653838 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.653633    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:19:10.655674 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.655650    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"]
Jul 17 18:19:10.730793 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.730746    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efeec5f5-cd52-4df6-af76-c73d7f30767b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-spsc4\" (UID: \"efeec5f5-cd52-4df6-af76-c73d7f30767b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"
Jul 17 18:19:10.832099 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.832057    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efeec5f5-cd52-4df6-af76-c73d7f30767b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-spsc4\" (UID: \"efeec5f5-cd52-4df6-af76-c73d7f30767b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"
Jul 17 18:19:10.832774 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.832755    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efeec5f5-cd52-4df6-af76-c73d7f30767b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-spsc4\" (UID: \"efeec5f5-cd52-4df6-af76-c73d7f30767b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"
Jul 17 18:19:10.961970 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:10.961873    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"
Jul 17 18:19:11.108590 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:11.108559    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"]
Jul 17 18:19:11.110386 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:19:11.110356    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefeec5f5_cd52_4df6_af76_c73d7f30767b.slice/crio-0067d8a4bb92da960f8ad2892f016077cafe39e52ad7ff2686d27c9c0c1e49db WatchSource:0}: Error finding container 0067d8a4bb92da960f8ad2892f016077cafe39e52ad7ff2686d27c9c0c1e49db: Status 404 returned error can't find the container with id 0067d8a4bb92da960f8ad2892f016077cafe39e52ad7ff2686d27c9c0c1e49db
Jul 17 18:19:11.418191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:11.418148    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4" event={"ID":"efeec5f5-cd52-4df6-af76-c73d7f30767b","Type":"ContainerStarted","Data":"0067d8a4bb92da960f8ad2892f016077cafe39e52ad7ff2686d27c9c0c1e49db"}
Jul 17 18:19:12.426376 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:12.426338    2572 generic.go:356] "Generic (PLEG): container finished" podID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerID="998cc7983f6bdc4862ae030ae151215f8ada290dc43c8eea691b5c1f7c49a1a4" exitCode=6
Jul 17 18:19:12.426837 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:12.426382    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4" event={"ID":"efeec5f5-cd52-4df6-af76-c73d7f30767b","Type":"ContainerDied","Data":"998cc7983f6bdc4862ae030ae151215f8ada290dc43c8eea691b5c1f7c49a1a4"}
Jul 17 18:19:12.426837 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:12.426659    2572 scope.go:117] "RemoveContainer" containerID="998cc7983f6bdc4862ae030ae151215f8ada290dc43c8eea691b5c1f7c49a1a4"
Jul 17 18:19:13.435378 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:13.435280    2572 generic.go:356] "Generic (PLEG): container finished" podID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerID="55c5ad6ab198036cdc2ac117227d35a7b657feaafbe010be71b7ad2e9487c7c5" exitCode=6
Jul 17 18:19:13.435378 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:13.435343    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4" event={"ID":"efeec5f5-cd52-4df6-af76-c73d7f30767b","Type":"ContainerDied","Data":"55c5ad6ab198036cdc2ac117227d35a7b657feaafbe010be71b7ad2e9487c7c5"}
Jul 17 18:19:13.435916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:13.435394    2572 scope.go:117] "RemoveContainer" containerID="998cc7983f6bdc4862ae030ae151215f8ada290dc43c8eea691b5c1f7c49a1a4"
Jul 17 18:19:13.435916 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:13.435631    2572 scope.go:117] "RemoveContainer" containerID="55c5ad6ab198036cdc2ac117227d35a7b657feaafbe010be71b7ad2e9487c7c5"
Jul 17 18:19:13.435916 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:13.435868    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-spsc4_odh-ai-gateway-infra(efeec5f5-cd52-4df6-af76-c73d7f30767b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b"
Jul 17 18:19:14.443162 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:14.443134    2572 scope.go:117] "RemoveContainer" containerID="55c5ad6ab198036cdc2ac117227d35a7b657feaafbe010be71b7ad2e9487c7c5"
Jul 17 18:19:14.443575 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:14.443321    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-spsc4_odh-ai-gateway-infra(efeec5f5-cd52-4df6-af76-c73d7f30767b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b"
Jul 17 18:19:17.105021 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.104981    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"]
Jul 17 18:19:17.105438 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.105333    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" containerName="revoke-keys"
Jul 17 18:19:17.105438 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.105344    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" containerName="revoke-keys"
Jul 17 18:19:17.105438 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.105364    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" containerName="revoke-keys"
Jul 17 18:19:17.105438 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.105370    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" containerName="revoke-keys"
Jul 17 18:19:17.105597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.105442    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc1dd56c-624d-470f-af39-08993a4c9673" containerName="revoke-keys"
Jul 17 18:19:17.105597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.105452    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9593bf8f-7084-4512-a714-be41ed03c85d" containerName="revoke-keys"
Jul 17 18:19:17.109804 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.109782    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"
Jul 17 18:19:17.117044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.117013    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"]
Jul 17 18:19:17.197450 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.197408    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40a3b07-70d8-4552-85e9-51f21980f365-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-mvspw\" (UID: \"a40a3b07-70d8-4552-85e9-51f21980f365\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"
Jul 17 18:19:17.298533 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.298462    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40a3b07-70d8-4552-85e9-51f21980f365-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-mvspw\" (UID: \"a40a3b07-70d8-4552-85e9-51f21980f365\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"
Jul 17 18:19:17.299154 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.299130    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40a3b07-70d8-4552-85e9-51f21980f365-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-mvspw\" (UID: \"a40a3b07-70d8-4552-85e9-51f21980f365\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"
Jul 17 18:19:17.421935 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.421836    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"
Jul 17 18:19:17.597022 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:17.596989    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"]
Jul 17 18:19:17.598594 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:19:17.598565    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40a3b07_70d8_4552_85e9_51f21980f365.slice/crio-38a927ba9abec412141411668f75f3eeb78084cd948ee154702d77eb8b1a1ad8 WatchSource:0}: Error finding container 38a927ba9abec412141411668f75f3eeb78084cd948ee154702d77eb8b1a1ad8: Status 404 returned error can't find the container with id 38a927ba9abec412141411668f75f3eeb78084cd948ee154702d77eb8b1a1ad8
Jul 17 18:19:18.467751 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:18.467712    2572 generic.go:356] "Generic (PLEG): container finished" podID="a40a3b07-70d8-4552-85e9-51f21980f365" containerID="9a94d4116df8fd39ff34f4ca08eb87f76ec5a2bef6aea3399f485bd31c4a2fc8" exitCode=6
Jul 17 18:19:18.468226 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:18.467778    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw" event={"ID":"a40a3b07-70d8-4552-85e9-51f21980f365","Type":"ContainerDied","Data":"9a94d4116df8fd39ff34f4ca08eb87f76ec5a2bef6aea3399f485bd31c4a2fc8"}
Jul 17 18:19:18.468226 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:18.467824    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw" event={"ID":"a40a3b07-70d8-4552-85e9-51f21980f365","Type":"ContainerStarted","Data":"38a927ba9abec412141411668f75f3eeb78084cd948ee154702d77eb8b1a1ad8"}
Jul 17 18:19:18.468226 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:18.468013    2572 scope.go:117] "RemoveContainer" containerID="9a94d4116df8fd39ff34f4ca08eb87f76ec5a2bef6aea3399f485bd31c4a2fc8"
Jul 17 18:19:19.008686 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.008597    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"]
Jul 17 18:19:19.012105 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.012086    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"
Jul 17 18:19:19.019785 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.019749    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"]
Jul 17 18:19:19.122914 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.122868    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f8e99d0-e660-49c9-9419-707cc80decd6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs\" (UID: \"3f8e99d0-e660-49c9-9419-707cc80decd6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"
Jul 17 18:19:19.224451 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.224408    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f8e99d0-e660-49c9-9419-707cc80decd6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs\" (UID: \"3f8e99d0-e660-49c9-9419-707cc80decd6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"
Jul 17 18:19:19.225155 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.225131    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f8e99d0-e660-49c9-9419-707cc80decd6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs\" (UID: \"3f8e99d0-e660-49c9-9419-707cc80decd6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"
Jul 17 18:19:19.325559 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.325519    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"
Jul 17 18:19:19.470345 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.470315    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"]
Jul 17 18:19:19.471053 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:19:19.471020    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f8e99d0_e660_49c9_9419_707cc80decd6.slice/crio-e0d84570731743cc9b81277d08fb8b220573e517b0016409e9f6693cb7056198 WatchSource:0}: Error finding container e0d84570731743cc9b81277d08fb8b220573e517b0016409e9f6693cb7056198: Status 404 returned error can't find the container with id e0d84570731743cc9b81277d08fb8b220573e517b0016409e9f6693cb7056198
Jul 17 18:19:19.476743 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.476714    2572 generic.go:356] "Generic (PLEG): container finished" podID="a40a3b07-70d8-4552-85e9-51f21980f365" containerID="8ecaf58426e7dc6bb7a928e2a4e514f2056705c6da6436a8b8d8036c4fd48c08" exitCode=6
Jul 17 18:19:19.476857 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.476794    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw" event={"ID":"a40a3b07-70d8-4552-85e9-51f21980f365","Type":"ContainerDied","Data":"8ecaf58426e7dc6bb7a928e2a4e514f2056705c6da6436a8b8d8036c4fd48c08"}
Jul 17 18:19:19.476911 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.476862    2572 scope.go:117] "RemoveContainer" containerID="9a94d4116df8fd39ff34f4ca08eb87f76ec5a2bef6aea3399f485bd31c4a2fc8"
Jul 17 18:19:19.477145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:19.477122    2572 scope.go:117] "RemoveContainer" containerID="8ecaf58426e7dc6bb7a928e2a4e514f2056705c6da6436a8b8d8036c4fd48c08"
Jul 17 18:19:19.477428 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:19.477398    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-mvspw_odh-ai-gateway-infra(a40a3b07-70d8-4552-85e9-51f21980f365)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw" podUID="a40a3b07-70d8-4552-85e9-51f21980f365"
Jul 17 18:19:20.484764 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:20.484723    2572 generic.go:356] "Generic (PLEG): container finished" podID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerID="46558bb145a0f922fb0d5def6cdc98b2eda1fdf10b828e02bcb741cedd7b5297" exitCode=6
Jul 17 18:19:20.485257 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:20.484809    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs" event={"ID":"3f8e99d0-e660-49c9-9419-707cc80decd6","Type":"ContainerDied","Data":"46558bb145a0f922fb0d5def6cdc98b2eda1fdf10b828e02bcb741cedd7b5297"}
Jul 17 18:19:20.485257 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:20.484873    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs" event={"ID":"3f8e99d0-e660-49c9-9419-707cc80decd6","Type":"ContainerStarted","Data":"e0d84570731743cc9b81277d08fb8b220573e517b0016409e9f6693cb7056198"}
Jul 17 18:19:20.485257 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:20.485032    2572 scope.go:117] "RemoveContainer" containerID="46558bb145a0f922fb0d5def6cdc98b2eda1fdf10b828e02bcb741cedd7b5297"
Jul 17 18:19:20.487424 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:20.487372    2572 scope.go:117] "RemoveContainer" containerID="8ecaf58426e7dc6bb7a928e2a4e514f2056705c6da6436a8b8d8036c4fd48c08"
Jul 17 18:19:20.487645 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:20.487607    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-mvspw_odh-ai-gateway-infra(a40a3b07-70d8-4552-85e9-51f21980f365)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw" podUID="a40a3b07-70d8-4552-85e9-51f21980f365"
Jul 17 18:19:21.494728 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:21.494624    2572 generic.go:356] "Generic (PLEG): container finished" podID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerID="ee26ad74da8141b693aa6b4571f487f776230208bf7320ba713d3b7844c3f4a6" exitCode=6
Jul 17 18:19:21.494728 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:21.494710    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs" event={"ID":"3f8e99d0-e660-49c9-9419-707cc80decd6","Type":"ContainerDied","Data":"ee26ad74da8141b693aa6b4571f487f776230208bf7320ba713d3b7844c3f4a6"}
Jul 17 18:19:21.495231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:21.494772    2572 scope.go:117] "RemoveContainer" containerID="46558bb145a0f922fb0d5def6cdc98b2eda1fdf10b828e02bcb741cedd7b5297"
Jul 17 18:19:21.495231 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:21.495011    2572 scope.go:117] "RemoveContainer" containerID="ee26ad74da8141b693aa6b4571f487f776230208bf7320ba713d3b7844c3f4a6"
Jul 17 18:19:21.495339 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:21.495229    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs_odh-ai-gateway-infra(3f8e99d0-e660-49c9-9419-707cc80decd6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6"
Jul 17 18:19:22.502619 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:22.502585    2572 scope.go:117] "RemoveContainer" containerID="ee26ad74da8141b693aa6b4571f487f776230208bf7320ba713d3b7844c3f4a6"
Jul 17 18:19:22.503094 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:22.503011    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs_odh-ai-gateway-infra(3f8e99d0-e660-49c9-9419-707cc80decd6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6"
Jul 17 18:19:26.510308 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:26.510256    2572 scope.go:117] "RemoveContainer" containerID="55c5ad6ab198036cdc2ac117227d35a7b657feaafbe010be71b7ad2e9487c7c5"
Jul 17 18:19:27.532271 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:27.532171    2572 generic.go:356] "Generic (PLEG): container finished" podID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerID="72e404c6167e5ebdbe06d0312bd2ef1609477af4a1af855d62f348b230b178de" exitCode=6
Jul 17 18:19:27.532271 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:27.532248    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4" event={"ID":"efeec5f5-cd52-4df6-af76-c73d7f30767b","Type":"ContainerDied","Data":"72e404c6167e5ebdbe06d0312bd2ef1609477af4a1af855d62f348b230b178de"}
Jul 17 18:19:27.532804 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:27.532326    2572 scope.go:117] "RemoveContainer" containerID="55c5ad6ab198036cdc2ac117227d35a7b657feaafbe010be71b7ad2e9487c7c5"
Jul 17 18:19:27.532804 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:27.532626    2572 scope.go:117] "RemoveContainer" containerID="72e404c6167e5ebdbe06d0312bd2ef1609477af4a1af855d62f348b230b178de"
Jul 17 18:19:27.532896 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:27.532828    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-spsc4_odh-ai-gateway-infra(efeec5f5-cd52-4df6-af76-c73d7f30767b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b"
Jul 17 18:19:28.562138 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:28.562098    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"]
Jul 17 18:19:28.694711 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:28.694685    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"
Jul 17 18:19:28.714470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:28.714439    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efeec5f5-cd52-4df6-af76-c73d7f30767b-maas-api-service-ca\") pod \"efeec5f5-cd52-4df6-af76-c73d7f30767b\" (UID: \"efeec5f5-cd52-4df6-af76-c73d7f30767b\") "
Jul 17 18:19:28.714869 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:28.714844    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efeec5f5-cd52-4df6-af76-c73d7f30767b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "efeec5f5-cd52-4df6-af76-c73d7f30767b" (UID: "efeec5f5-cd52-4df6-af76-c73d7f30767b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:19:28.815349 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:28.815251    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efeec5f5-cd52-4df6-af76-c73d7f30767b-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:19:29.562019 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:29.561988    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"
Jul 17 18:19:29.562278 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:29.562024    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4" event={"ID":"efeec5f5-cd52-4df6-af76-c73d7f30767b","Type":"ContainerDied","Data":"0067d8a4bb92da960f8ad2892f016077cafe39e52ad7ff2686d27c9c0c1e49db"}
Jul 17 18:19:29.562278 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:29.562084    2572 scope.go:117] "RemoveContainer" containerID="72e404c6167e5ebdbe06d0312bd2ef1609477af4a1af855d62f348b230b178de"
Jul 17 18:19:29.591215 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:29.591173    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"]
Jul 17 18:19:29.595999 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:29.595961    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-spsc4"]
Jul 17 18:19:30.515481 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:30.515446    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" path="/var/lib/kubelet/pods/efeec5f5-cd52-4df6-af76-c73d7f30767b/volumes"
Jul 17 18:19:34.905256 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.905218    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"]
Jul 17 18:19:34.905699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.905593    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerName="revoke-keys"
Jul 17 18:19:34.905699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.905605    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerName="revoke-keys"
Jul 17 18:19:34.905699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.905630    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerName="revoke-keys"
Jul 17 18:19:34.905699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.905635    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerName="revoke-keys"
Jul 17 18:19:34.905699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.905692    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerName="revoke-keys"
Jul 17 18:19:34.905699 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.905701    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerName="revoke-keys"
Jul 17 18:19:34.905907 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.905707    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerName="revoke-keys"
Jul 17 18:19:34.908744 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.908725    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"
Jul 17 18:19:34.918043 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.918008    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"]
Jul 17 18:19:34.961428 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:34.961381    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6ff279-69e4-4b79-a15c-7773e490aecf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf\" (UID: \"8b6ff279-69e4-4b79-a15c-7773e490aecf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"
Jul 17 18:19:35.066512 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:35.063028    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6ff279-69e4-4b79-a15c-7773e490aecf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf\" (UID: \"8b6ff279-69e4-4b79-a15c-7773e490aecf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"
Jul 17 18:19:35.066512 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:35.064035    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6ff279-69e4-4b79-a15c-7773e490aecf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf\" (UID: \"8b6ff279-69e4-4b79-a15c-7773e490aecf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"
Jul 17 18:19:35.220870 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:35.220767    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"
Jul 17 18:19:35.370324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:35.370283    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"]
Jul 17 18:19:35.371639 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:19:35.371609    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6ff279_69e4_4b79_a15c_7773e490aecf.slice/crio-21cd703478f66bd0d35638c3e6f325b1acc8dc66f9c12fa76f7975ae21eaed70 WatchSource:0}: Error finding container 21cd703478f66bd0d35638c3e6f325b1acc8dc66f9c12fa76f7975ae21eaed70: Status 404 returned error can't find the container with id 21cd703478f66bd0d35638c3e6f325b1acc8dc66f9c12fa76f7975ae21eaed70
Jul 17 18:19:35.510338 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:35.510240    2572 scope.go:117] "RemoveContainer" containerID="8ecaf58426e7dc6bb7a928e2a4e514f2056705c6da6436a8b8d8036c4fd48c08"
Jul 17 18:19:35.597440 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:35.597397    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf" event={"ID":"8b6ff279-69e4-4b79-a15c-7773e490aecf","Type":"ContainerStarted","Data":"21cd703478f66bd0d35638c3e6f325b1acc8dc66f9c12fa76f7975ae21eaed70"}
Jul 17 18:19:36.515728 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:36.515652    2572 scope.go:117] "RemoveContainer" containerID="ee26ad74da8141b693aa6b4571f487f776230208bf7320ba713d3b7844c3f4a6"
Jul 17 18:19:36.605473 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:36.605435    2572 generic.go:356] "Generic (PLEG): container finished" podID="a40a3b07-70d8-4552-85e9-51f21980f365" containerID="16c88a4d996b567aa5fed72cfb88c6832e212de288cf34f1c213193563e9663f" exitCode=6
Jul 17 18:19:36.605688 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:36.605537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw" event={"ID":"a40a3b07-70d8-4552-85e9-51f21980f365","Type":"ContainerDied","Data":"16c88a4d996b567aa5fed72cfb88c6832e212de288cf34f1c213193563e9663f"}
Jul 17 18:19:36.605688 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:36.605615    2572 scope.go:117] "RemoveContainer" containerID="8ecaf58426e7dc6bb7a928e2a4e514f2056705c6da6436a8b8d8036c4fd48c08"
Jul 17 18:19:36.605967 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:36.605946    2572 scope.go:117] "RemoveContainer" containerID="16c88a4d996b567aa5fed72cfb88c6832e212de288cf34f1c213193563e9663f"
Jul 17 18:19:36.606193 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:36.606174    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-mvspw_odh-ai-gateway-infra(a40a3b07-70d8-4552-85e9-51f21980f365)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw" podUID="a40a3b07-70d8-4552-85e9-51f21980f365"
Jul 17 18:19:36.607271 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:36.607248    2572 generic.go:356] "Generic (PLEG): container finished" podID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerID="6a6b7588ec0415c235c445e61302f541cf251b933f9b43c77efa6d2fb1a3b43e" exitCode=6
Jul 17 18:19:36.607375 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:36.607327    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf" event={"ID":"8b6ff279-69e4-4b79-a15c-7773e490aecf","Type":"ContainerDied","Data":"6a6b7588ec0415c235c445e61302f541cf251b933f9b43c77efa6d2fb1a3b43e"}
Jul 17 18:19:36.607537 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:36.607523    2572 scope.go:117] "RemoveContainer" containerID="6a6b7588ec0415c235c445e61302f541cf251b933f9b43c77efa6d2fb1a3b43e"
Jul 17 18:19:37.615255 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.615172    2572 generic.go:356] "Generic (PLEG): container finished" podID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerID="dbfe125a46562bccd1971833896c9c8889fdeee4e5ffa9f21df24705d3d1b4a1" exitCode=6
Jul 17 18:19:37.615255 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.615246    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs" event={"ID":"3f8e99d0-e660-49c9-9419-707cc80decd6","Type":"ContainerDied","Data":"dbfe125a46562bccd1971833896c9c8889fdeee4e5ffa9f21df24705d3d1b4a1"}
Jul 17 18:19:37.615816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.615308    2572 scope.go:117] "RemoveContainer" containerID="ee26ad74da8141b693aa6b4571f487f776230208bf7320ba713d3b7844c3f4a6"
Jul 17 18:19:37.615816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.615669    2572 scope.go:117] "RemoveContainer" containerID="dbfe125a46562bccd1971833896c9c8889fdeee4e5ffa9f21df24705d3d1b4a1"
Jul 17 18:19:37.615927 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:37.615888    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs_odh-ai-gateway-infra(3f8e99d0-e660-49c9-9419-707cc80decd6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6"
Jul 17 18:19:37.618651 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.618626    2572 generic.go:356] "Generic (PLEG): container finished" podID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerID="1014ec08c4d43571fa5541d7985f61db40177aa72b0132710ab9d4ff5657c0cf" exitCode=6
Jul 17 18:19:37.618764 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.618681    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf" event={"ID":"8b6ff279-69e4-4b79-a15c-7773e490aecf","Type":"ContainerDied","Data":"1014ec08c4d43571fa5541d7985f61db40177aa72b0132710ab9d4ff5657c0cf"}
Jul 17 18:19:37.618891 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.618878    2572 scope.go:117] "RemoveContainer" containerID="1014ec08c4d43571fa5541d7985f61db40177aa72b0132710ab9d4ff5657c0cf"
Jul 17 18:19:37.619088 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:37.619071    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf_odh-ai-gateway-infra(8b6ff279-69e4-4b79-a15c-7773e490aecf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf"
Jul 17 18:19:37.628651 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.628624    2572 scope.go:117] "RemoveContainer" containerID="6a6b7588ec0415c235c445e61302f541cf251b933f9b43c77efa6d2fb1a3b43e"
Jul 17 18:19:37.636190 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.636159    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"]
Jul 17 18:19:37.781547 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.781516    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"
Jul 17 18:19:37.889835 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.889742    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40a3b07-70d8-4552-85e9-51f21980f365-maas-api-service-ca\") pod \"a40a3b07-70d8-4552-85e9-51f21980f365\" (UID: \"a40a3b07-70d8-4552-85e9-51f21980f365\") "
Jul 17 18:19:37.890133 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.890109    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a40a3b07-70d8-4552-85e9-51f21980f365-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a40a3b07-70d8-4552-85e9-51f21980f365" (UID: "a40a3b07-70d8-4552-85e9-51f21980f365"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:19:37.990310 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:37.990261    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40a3b07-70d8-4552-85e9-51f21980f365-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:19:38.628047 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.628018    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"
Jul 17 18:19:38.628047 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.628026    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw" event={"ID":"a40a3b07-70d8-4552-85e9-51f21980f365","Type":"ContainerDied","Data":"38a927ba9abec412141411668f75f3eeb78084cd948ee154702d77eb8b1a1ad8"}
Jul 17 18:19:38.628658 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.628089    2572 scope.go:117] "RemoveContainer" containerID="16c88a4d996b567aa5fed72cfb88c6832e212de288cf34f1c213193563e9663f"
Jul 17 18:19:38.630518 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.630481    2572 scope.go:117] "RemoveContainer" containerID="1014ec08c4d43571fa5541d7985f61db40177aa72b0132710ab9d4ff5657c0cf"
Jul 17 18:19:38.630963 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:38.630934    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf_odh-ai-gateway-infra(8b6ff279-69e4-4b79-a15c-7773e490aecf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf"
Jul 17 18:19:38.665288 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.665247    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"]
Jul 17 18:19:38.712722 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.712675    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"]
Jul 17 18:19:38.717357 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.717321    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-mvspw"]
Jul 17 18:19:38.816334 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.816298    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"
Jul 17 18:19:38.900396 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.900344    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f8e99d0-e660-49c9-9419-707cc80decd6-maas-api-service-ca\") pod \"3f8e99d0-e660-49c9-9419-707cc80decd6\" (UID: \"3f8e99d0-e660-49c9-9419-707cc80decd6\") "
Jul 17 18:19:38.900770 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:38.900748    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f8e99d0-e660-49c9-9419-707cc80decd6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f8e99d0-e660-49c9-9419-707cc80decd6" (UID: "3f8e99d0-e660-49c9-9419-707cc80decd6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:19:39.001831 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:39.001777    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f8e99d0-e660-49c9-9419-707cc80decd6-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:19:39.637124 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:39.637086    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"
Jul 17 18:19:39.637124 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:39.637097    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs" event={"ID":"3f8e99d0-e660-49c9-9419-707cc80decd6","Type":"ContainerDied","Data":"e0d84570731743cc9b81277d08fb8b220573e517b0016409e9f6693cb7056198"}
Jul 17 18:19:39.637731 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:39.637169    2572 scope.go:117] "RemoveContainer" containerID="dbfe125a46562bccd1971833896c9c8889fdeee4e5ffa9f21df24705d3d1b4a1"
Jul 17 18:19:39.669821 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:39.669784    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"]
Jul 17 18:19:39.673633 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:39.673603    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-6gfbs"]
Jul 17 18:19:40.516075 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:40.516028    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" path="/var/lib/kubelet/pods/3f8e99d0-e660-49c9-9419-707cc80decd6/volumes"
Jul 17 18:19:40.516463 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:40.516441    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" path="/var/lib/kubelet/pods/a40a3b07-70d8-4552-85e9-51f21980f365/volumes"
Jul 17 18:19:49.510141 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:49.510108    2572 scope.go:117] "RemoveContainer" containerID="1014ec08c4d43571fa5541d7985f61db40177aa72b0132710ab9d4ff5657c0cf"
Jul 17 18:19:50.704929 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:50.704889    2572 generic.go:356] "Generic (PLEG): container finished" podID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerID="b2ff27f2edece87face11d4e0f951d712657205e83ffc0734ccd02bb5c28d14c" exitCode=6
Jul 17 18:19:50.705448 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:50.704961    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf" event={"ID":"8b6ff279-69e4-4b79-a15c-7773e490aecf","Type":"ContainerDied","Data":"b2ff27f2edece87face11d4e0f951d712657205e83ffc0734ccd02bb5c28d14c"}
Jul 17 18:19:50.705448 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:50.705026    2572 scope.go:117] "RemoveContainer" containerID="1014ec08c4d43571fa5541d7985f61db40177aa72b0132710ab9d4ff5657c0cf"
Jul 17 18:19:50.705448 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:50.705342    2572 scope.go:117] "RemoveContainer" containerID="b2ff27f2edece87face11d4e0f951d712657205e83ffc0734ccd02bb5c28d14c"
Jul 17 18:19:50.705672 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:19:50.705646    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf_odh-ai-gateway-infra(8b6ff279-69e4-4b79-a15c-7773e490aecf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf"
Jul 17 18:19:51.733348 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:51.733308    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"]
Jul 17 18:19:51.885274 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:51.885249    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"
Jul 17 18:19:52.023784 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:52.023666    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6ff279-69e4-4b79-a15c-7773e490aecf-maas-api-service-ca\") pod \"8b6ff279-69e4-4b79-a15c-7773e490aecf\" (UID: \"8b6ff279-69e4-4b79-a15c-7773e490aecf\") "
Jul 17 18:19:52.024073 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:52.024046    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b6ff279-69e4-4b79-a15c-7773e490aecf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8b6ff279-69e4-4b79-a15c-7773e490aecf" (UID: "8b6ff279-69e4-4b79-a15c-7773e490aecf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:19:52.124600 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:52.124559    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6ff279-69e4-4b79-a15c-7773e490aecf-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:19:52.719649 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:52.719620    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"
Jul 17 18:19:52.719649 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:52.719639    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf" event={"ID":"8b6ff279-69e4-4b79-a15c-7773e490aecf","Type":"ContainerDied","Data":"21cd703478f66bd0d35638c3e6f325b1acc8dc66f9c12fa76f7975ae21eaed70"}
Jul 17 18:19:52.719933 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:52.719704    2572 scope.go:117] "RemoveContainer" containerID="b2ff27f2edece87face11d4e0f951d712657205e83ffc0734ccd02bb5c28d14c"
Jul 17 18:19:52.738062 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:52.738029    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"]
Jul 17 18:19:52.744293 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:52.744262    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-jx4gf"]
Jul 17 18:19:54.515562 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:19:54.515522    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" path="/var/lib/kubelet/pods/8b6ff279-69e4-4b79-a15c-7773e490aecf/volumes"
Jul 17 18:20:00.685331 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685292    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"]
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685683    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685697    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685708    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685713    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685721    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685726    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685734    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685740    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685747    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685752    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efeec5f5-cd52-4df6-af76-c73d7f30767b" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685761    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685767    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685780    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685785    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685791    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerName="revoke-keys"
Jul 17 18:20:00.685816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685796    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerName="revoke-keys"
Jul 17 18:20:00.686302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685850    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" containerName="revoke-keys"
Jul 17 18:20:00.686302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685859    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" containerName="revoke-keys"
Jul 17 18:20:00.686302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685865    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" containerName="revoke-keys"
Jul 17 18:20:00.686302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685870    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerName="revoke-keys"
Jul 17 18:20:00.686302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685878    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerName="revoke-keys"
Jul 17 18:20:00.686302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685883    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerName="revoke-keys"
Jul 17 18:20:00.686302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685889    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerName="revoke-keys"
Jul 17 18:20:00.686302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685894    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerName="revoke-keys"
Jul 17 18:20:00.686302 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.685900    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b6ff279-69e4-4b79-a15c-7773e490aecf" containerName="revoke-keys"
Jul 17 18:20:00.689132 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.689115    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"
Jul 17 18:20:00.692345 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.692317    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:20:00.692874 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.692837    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:20:00.697944 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.697641    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"]
Jul 17 18:20:00.805855 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.805804    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0409c5c-2d27-4cb1-b781-537f729844cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-drfkj\" (UID: \"d0409c5c-2d27-4cb1-b781-537f729844cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"
Jul 17 18:20:00.907075 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.907036    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0409c5c-2d27-4cb1-b781-537f729844cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-drfkj\" (UID: \"d0409c5c-2d27-4cb1-b781-537f729844cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"
Jul 17 18:20:00.907816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:00.907793    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0409c5c-2d27-4cb1-b781-537f729844cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-drfkj\" (UID: \"d0409c5c-2d27-4cb1-b781-537f729844cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"
Jul 17 18:20:01.003055 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:01.002954    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"
Jul 17 18:20:01.145363 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:01.145336    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"]
Jul 17 18:20:01.147625 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:20:01.147587    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0409c5c_2d27_4cb1_b781_537f729844cd.slice/crio-87a113187a446a678d3ab5da9e864d2826460f2c479531cebfb62e7a1b7b8ca9 WatchSource:0}: Error finding container 87a113187a446a678d3ab5da9e864d2826460f2c479531cebfb62e7a1b7b8ca9: Status 404 returned error can't find the container with id 87a113187a446a678d3ab5da9e864d2826460f2c479531cebfb62e7a1b7b8ca9
Jul 17 18:20:01.774175 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:01.774135    2572 generic.go:356] "Generic (PLEG): container finished" podID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerID="3dfb61ad04d0c9ef09364885a7e3cb486d670fcfd8dc2e563e144cf6b2e50b22" exitCode=6
Jul 17 18:20:01.774656 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:01.774182    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" event={"ID":"d0409c5c-2d27-4cb1-b781-537f729844cd","Type":"ContainerDied","Data":"3dfb61ad04d0c9ef09364885a7e3cb486d670fcfd8dc2e563e144cf6b2e50b22"}
Jul 17 18:20:01.774656 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:01.774230    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" event={"ID":"d0409c5c-2d27-4cb1-b781-537f729844cd","Type":"ContainerStarted","Data":"87a113187a446a678d3ab5da9e864d2826460f2c479531cebfb62e7a1b7b8ca9"}
Jul 17 18:20:01.774656 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:01.774425    2572 scope.go:117] "RemoveContainer" containerID="3dfb61ad04d0c9ef09364885a7e3cb486d670fcfd8dc2e563e144cf6b2e50b22"
Jul 17 18:20:02.782158 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:02.782059    2572 generic.go:356] "Generic (PLEG): container finished" podID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerID="0610794286bda4c967dae89a438e58a837727355b186672530df7ece6adf6591" exitCode=6
Jul 17 18:20:02.782718 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:02.782150    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" event={"ID":"d0409c5c-2d27-4cb1-b781-537f729844cd","Type":"ContainerDied","Data":"0610794286bda4c967dae89a438e58a837727355b186672530df7ece6adf6591"}
Jul 17 18:20:02.782718 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:02.782207    2572 scope.go:117] "RemoveContainer" containerID="3dfb61ad04d0c9ef09364885a7e3cb486d670fcfd8dc2e563e144cf6b2e50b22"
Jul 17 18:20:02.782718 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:02.782333    2572 scope.go:117] "RemoveContainer" containerID="0610794286bda4c967dae89a438e58a837727355b186672530df7ece6adf6591"
Jul 17 18:20:02.782718 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:02.782620    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-drfkj_odh-ai-gateway-infra(d0409c5c-2d27-4cb1-b781-537f729844cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd"
Jul 17 18:20:03.790918 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:03.790888    2572 scope.go:117] "RemoveContainer" containerID="0610794286bda4c967dae89a438e58a837727355b186672530df7ece6adf6591"
Jul 17 18:20:03.791337 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:03.791111    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-drfkj_odh-ai-gateway-infra(d0409c5c-2d27-4cb1-b781-537f729844cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd"
Jul 17 18:20:16.510625 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:16.510589    2572 scope.go:117] "RemoveContainer" containerID="0610794286bda4c967dae89a438e58a837727355b186672530df7ece6adf6591"
Jul 17 18:20:16.866990 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:16.866951    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" event={"ID":"d0409c5c-2d27-4cb1-b781-537f729844cd","Type":"ContainerStarted","Data":"4f920f86ec289cc9dd53714be755157a1d50aecfdc931dcbdfdabaeaaa7015a6"}
Jul 17 18:20:16.885694 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:16.885628    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" podStartSLOduration=16.885611304 podStartE2EDuration="16.885611304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:20:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:20:16.883193252 +0000 UTC m=+3388.888549107" watchObservedRunningTime="2026-07-17 18:20:16.885611304 +0000 UTC m=+3388.890967181"
Jul 17 18:20:17.146924 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.146892    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"]
Jul 17 18:20:17.147316 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.147299    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerName="revoke-keys"
Jul 17 18:20:17.147409 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.147319    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f8e99d0-e660-49c9-9419-707cc80decd6" containerName="revoke-keys"
Jul 17 18:20:17.147409 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.147344    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" containerName="revoke-keys"
Jul 17 18:20:17.147409 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.147353    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40a3b07-70d8-4552-85e9-51f21980f365" containerName="revoke-keys"
Jul 17 18:20:17.150425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.150402    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"
Jul 17 18:20:17.157797 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.157728    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"]
Jul 17 18:20:17.258675 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.258639    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aafc40a-e7b2-4db5-b0c3-2d9adf47618a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c\" (UID: \"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"
Jul 17 18:20:17.359141 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.359096    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aafc40a-e7b2-4db5-b0c3-2d9adf47618a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c\" (UID: \"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"
Jul 17 18:20:17.359803 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.359781    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aafc40a-e7b2-4db5-b0c3-2d9adf47618a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c\" (UID: \"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"
Jul 17 18:20:17.496368 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.496247    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"
Jul 17 18:20:17.649603 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.649575    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"]
Jul 17 18:20:17.650776 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:20:17.650745    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aafc40a_e7b2_4db5_b0c3_2d9adf47618a.slice/crio-c00811cfa15b314148353a4fdad8b9771889e567e63e0fe705adbec07db2247a WatchSource:0}: Error finding container c00811cfa15b314148353a4fdad8b9771889e567e63e0fe705adbec07db2247a: Status 404 returned error can't find the container with id c00811cfa15b314148353a4fdad8b9771889e567e63e0fe705adbec07db2247a
Jul 17 18:20:17.874662 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.874626    2572 generic.go:356] "Generic (PLEG): container finished" podID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerID="4f920f86ec289cc9dd53714be755157a1d50aecfdc931dcbdfdabaeaaa7015a6" exitCode=6
Jul 17 18:20:17.874858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.874702    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" event={"ID":"d0409c5c-2d27-4cb1-b781-537f729844cd","Type":"ContainerDied","Data":"4f920f86ec289cc9dd53714be755157a1d50aecfdc931dcbdfdabaeaaa7015a6"}
Jul 17 18:20:17.874858 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.874781    2572 scope.go:117] "RemoveContainer" containerID="0610794286bda4c967dae89a438e58a837727355b186672530df7ece6adf6591"
Jul 17 18:20:17.875186 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.875162    2572 scope.go:117] "RemoveContainer" containerID="4f920f86ec289cc9dd53714be755157a1d50aecfdc931dcbdfdabaeaaa7015a6"
Jul 17 18:20:17.875683 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:17.875430    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-drfkj_odh-ai-gateway-infra(d0409c5c-2d27-4cb1-b781-537f729844cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd"
Jul 17 18:20:17.876213 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.876189    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" event={"ID":"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a","Type":"ContainerStarted","Data":"c00811cfa15b314148353a4fdad8b9771889e567e63e0fe705adbec07db2247a"}
Jul 17 18:20:17.901589 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:17.901553    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"]
Jul 17 18:20:18.885553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:18.885526    2572 generic.go:356] "Generic (PLEG): container finished" podID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerID="3fd22c43885892d64ad912a92579bbb843b11c8e50a61bb141777ff120f6c024" exitCode=6
Jul 17 18:20:18.885894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:18.885571    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" event={"ID":"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a","Type":"ContainerDied","Data":"3fd22c43885892d64ad912a92579bbb843b11c8e50a61bb141777ff120f6c024"}
Jul 17 18:20:18.885894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:18.885778    2572 scope.go:117] "RemoveContainer" containerID="3fd22c43885892d64ad912a92579bbb843b11c8e50a61bb141777ff120f6c024"
Jul 17 18:20:19.017349 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.017323    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"
Jul 17 18:20:19.067001 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.066962    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"]
Jul 17 18:20:19.067316 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.067304    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerName="revoke-keys"
Jul 17 18:20:19.067365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.067318    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerName="revoke-keys"
Jul 17 18:20:19.067365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.067340    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerName="revoke-keys"
Jul 17 18:20:19.067365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.067346    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerName="revoke-keys"
Jul 17 18:20:19.067365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.067353    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerName="revoke-keys"
Jul 17 18:20:19.067365 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.067359    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerName="revoke-keys"
Jul 17 18:20:19.068077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.068051    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerName="revoke-keys"
Jul 17 18:20:19.068077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.068078    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerName="revoke-keys"
Jul 17 18:20:19.068258 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.068108    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" containerName="revoke-keys"
Jul 17 18:20:19.072960 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.072930    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"
Jul 17 18:20:19.080051 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.080019    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"]
Jul 17 18:20:19.174241 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.174200    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0409c5c-2d27-4cb1-b781-537f729844cd-maas-api-service-ca\") pod \"d0409c5c-2d27-4cb1-b781-537f729844cd\" (UID: \"d0409c5c-2d27-4cb1-b781-537f729844cd\") "
Jul 17 18:20:19.174444 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.174377    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0844fe0-fdb2-4063-929d-698cce105dcd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl\" (UID: \"e0844fe0-fdb2-4063-929d-698cce105dcd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"
Jul 17 18:20:19.174626 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.174602    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0409c5c-2d27-4cb1-b781-537f729844cd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d0409c5c-2d27-4cb1-b781-537f729844cd" (UID: "d0409c5c-2d27-4cb1-b781-537f729844cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:20:19.275111 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.275066    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0844fe0-fdb2-4063-929d-698cce105dcd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl\" (UID: \"e0844fe0-fdb2-4063-929d-698cce105dcd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"
Jul 17 18:20:19.275301 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.275140    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0409c5c-2d27-4cb1-b781-537f729844cd-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:20:19.275839 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.275814    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0844fe0-fdb2-4063-929d-698cce105dcd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl\" (UID: \"e0844fe0-fdb2-4063-929d-698cce105dcd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"
Jul 17 18:20:19.386057 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.385961    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"
Jul 17 18:20:19.530939 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.530906    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"]
Jul 17 18:20:19.532323 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:20:19.532296    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0844fe0_fdb2_4063_929d_698cce105dcd.slice/crio-eef55f66e3129ae572fc334d7d8450e846fc434becb88bed3da60415ebcfb09b WatchSource:0}: Error finding container eef55f66e3129ae572fc334d7d8450e846fc434becb88bed3da60415ebcfb09b: Status 404 returned error can't find the container with id eef55f66e3129ae572fc334d7d8450e846fc434becb88bed3da60415ebcfb09b
Jul 17 18:20:19.892232 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.892195    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"
Jul 17 18:20:19.892232 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.892210    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj" event={"ID":"d0409c5c-2d27-4cb1-b781-537f729844cd","Type":"ContainerDied","Data":"87a113187a446a678d3ab5da9e864d2826460f2c479531cebfb62e7a1b7b8ca9"}
Jul 17 18:20:19.892813 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.892281    2572 scope.go:117] "RemoveContainer" containerID="4f920f86ec289cc9dd53714be755157a1d50aecfdc931dcbdfdabaeaaa7015a6"
Jul 17 18:20:19.894307 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.894274    2572 generic.go:356] "Generic (PLEG): container finished" podID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerID="2ae637d0f34664c7b606669d0c6bb1b1259270868de0b7232bc212abc964e466" exitCode=6
Jul 17 18:20:19.894410 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.894357    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" event={"ID":"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a","Type":"ContainerDied","Data":"2ae637d0f34664c7b606669d0c6bb1b1259270868de0b7232bc212abc964e466"}
Jul 17 18:20:19.894611 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.894590    2572 scope.go:117] "RemoveContainer" containerID="2ae637d0f34664c7b606669d0c6bb1b1259270868de0b7232bc212abc964e466"
Jul 17 18:20:19.894825 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:19.894794    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c_odh-ai-gateway-infra(4aafc40a-e7b2-4db5-b0c3-2d9adf47618a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a"
Jul 17 18:20:19.896225 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.896167    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" event={"ID":"e0844fe0-fdb2-4063-929d-698cce105dcd","Type":"ContainerStarted","Data":"b3f48c1a5424153fa79208949737062f9a6521e8648b8425fcc180a95ad161c1"}
Jul 17 18:20:19.896225 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.896220    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" event={"ID":"e0844fe0-fdb2-4063-929d-698cce105dcd","Type":"ContainerStarted","Data":"eef55f66e3129ae572fc334d7d8450e846fc434becb88bed3da60415ebcfb09b"}
Jul 17 18:20:19.902727 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.902709    2572 scope.go:117] "RemoveContainer" containerID="3fd22c43885892d64ad912a92579bbb843b11c8e50a61bb141777ff120f6c024"
Jul 17 18:20:19.930702 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.930650    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" podStartSLOduration=0.930632915 podStartE2EDuration="930.632915ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:20:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:20:19.928054515 +0000 UTC m=+3391.933410374" watchObservedRunningTime="2026-07-17 18:20:19.930632915 +0000 UTC m=+3391.935988769"
Jul 17 18:20:19.943239 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.943196    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"]
Jul 17 18:20:19.950063 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:19.950035    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-drfkj"]
Jul 17 18:20:20.516187 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:20.516103    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0409c5c-2d27-4cb1-b781-537f729844cd" path="/var/lib/kubelet/pods/d0409c5c-2d27-4cb1-b781-537f729844cd/volumes"
Jul 17 18:20:20.904065 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:20.904032    2572 generic.go:356] "Generic (PLEG): container finished" podID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerID="b3f48c1a5424153fa79208949737062f9a6521e8648b8425fcc180a95ad161c1" exitCode=6
Jul 17 18:20:20.904531 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:20.904117    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" event={"ID":"e0844fe0-fdb2-4063-929d-698cce105dcd","Type":"ContainerDied","Data":"b3f48c1a5424153fa79208949737062f9a6521e8648b8425fcc180a95ad161c1"}
Jul 17 18:20:20.904531 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:20.904342    2572 scope.go:117] "RemoveContainer" containerID="b3f48c1a5424153fa79208949737062f9a6521e8648b8425fcc180a95ad161c1"
Jul 17 18:20:20.907583 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:20.907551    2572 scope.go:117] "RemoveContainer" containerID="2ae637d0f34664c7b606669d0c6bb1b1259270868de0b7232bc212abc964e466"
Jul 17 18:20:20.907773 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:20.907753    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c_odh-ai-gateway-infra(4aafc40a-e7b2-4db5-b0c3-2d9adf47618a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a"
Jul 17 18:20:21.914817 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:21.914781    2572 generic.go:356] "Generic (PLEG): container finished" podID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerID="d3a7daab2bdfa57d8bc4ca350641e5f06c735e0d116d08188b659937419a805a" exitCode=6
Jul 17 18:20:21.915266 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:21.914843    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" event={"ID":"e0844fe0-fdb2-4063-929d-698cce105dcd","Type":"ContainerDied","Data":"d3a7daab2bdfa57d8bc4ca350641e5f06c735e0d116d08188b659937419a805a"}
Jul 17 18:20:21.915266 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:21.914900    2572 scope.go:117] "RemoveContainer" containerID="b3f48c1a5424153fa79208949737062f9a6521e8648b8425fcc180a95ad161c1"
Jul 17 18:20:21.915266 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:21.915067    2572 scope.go:117] "RemoveContainer" containerID="d3a7daab2bdfa57d8bc4ca350641e5f06c735e0d116d08188b659937419a805a"
Jul 17 18:20:21.915399 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:21.915323    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl_odh-ai-gateway-infra(e0844fe0-fdb2-4063-929d-698cce105dcd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd"
Jul 17 18:20:22.923409 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:22.923381    2572 scope.go:117] "RemoveContainer" containerID="d3a7daab2bdfa57d8bc4ca350641e5f06c735e0d116d08188b659937419a805a"
Jul 17 18:20:22.923826 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:22.923617    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl_odh-ai-gateway-infra(e0844fe0-fdb2-4063-929d-698cce105dcd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd"
Jul 17 18:20:24.946604 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:24.946562    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"]
Jul 17 18:20:24.951433 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:24.951407    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"
Jul 17 18:20:24.957409 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:24.957373    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"]
Jul 17 18:20:25.021680 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:25.021626    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7947789-0533-4755-beb6-3b5e7b58ad00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q\" (UID: \"a7947789-0533-4755-beb6-3b5e7b58ad00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"
Jul 17 18:20:25.123029 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:25.122978    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7947789-0533-4755-beb6-3b5e7b58ad00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q\" (UID: \"a7947789-0533-4755-beb6-3b5e7b58ad00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"
Jul 17 18:20:25.123693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:25.123665    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7947789-0533-4755-beb6-3b5e7b58ad00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q\" (UID: \"a7947789-0533-4755-beb6-3b5e7b58ad00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"
Jul 17 18:20:25.263851 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:25.263758    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"
Jul 17 18:20:25.414730 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:25.414704    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"]
Jul 17 18:20:25.416814 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:20:25.416779    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7947789_0533_4755_beb6_3b5e7b58ad00.slice/crio-32cbff7604a6af19ea655459dd19826b28918a242a90da8e7070ade1e149cfe2 WatchSource:0}: Error finding container 32cbff7604a6af19ea655459dd19826b28918a242a90da8e7070ade1e149cfe2: Status 404 returned error can't find the container with id 32cbff7604a6af19ea655459dd19826b28918a242a90da8e7070ade1e149cfe2
Jul 17 18:20:25.941684 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:25.941582    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" event={"ID":"a7947789-0533-4755-beb6-3b5e7b58ad00","Type":"ContainerStarted","Data":"30d24fbb3de682424e95a56fad4946c155d3eb9c6f66876f52c42d1f87226a5b"}
Jul 17 18:20:25.941684 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:25.941644    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" event={"ID":"a7947789-0533-4755-beb6-3b5e7b58ad00","Type":"ContainerStarted","Data":"32cbff7604a6af19ea655459dd19826b28918a242a90da8e7070ade1e149cfe2"}
Jul 17 18:20:25.960964 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:25.960903    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" podStartSLOduration=1.960886356 podStartE2EDuration="1.960886356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:20:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:20:25.960780868 +0000 UTC m=+3397.966136723" watchObservedRunningTime="2026-07-17 18:20:25.960886356 +0000 UTC m=+3397.966242212"
Jul 17 18:20:26.949305 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:26.949266    2572 generic.go:356] "Generic (PLEG): container finished" podID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerID="30d24fbb3de682424e95a56fad4946c155d3eb9c6f66876f52c42d1f87226a5b" exitCode=6
Jul 17 18:20:26.949538 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:26.949351    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" event={"ID":"a7947789-0533-4755-beb6-3b5e7b58ad00","Type":"ContainerDied","Data":"30d24fbb3de682424e95a56fad4946c155d3eb9c6f66876f52c42d1f87226a5b"}
Jul 17 18:20:26.949601 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:26.949566    2572 scope.go:117] "RemoveContainer" containerID="30d24fbb3de682424e95a56fad4946c155d3eb9c6f66876f52c42d1f87226a5b"
Jul 17 18:20:27.957681 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:27.957573    2572 generic.go:356] "Generic (PLEG): container finished" podID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerID="f29f9fb560f9b5d9f9f26a0c2712811388f2fe2379a179cc5236e356a0a4d009" exitCode=6
Jul 17 18:20:27.957681 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:27.957655    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" event={"ID":"a7947789-0533-4755-beb6-3b5e7b58ad00","Type":"ContainerDied","Data":"f29f9fb560f9b5d9f9f26a0c2712811388f2fe2379a179cc5236e356a0a4d009"}
Jul 17 18:20:27.958221 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:27.957717    2572 scope.go:117] "RemoveContainer" containerID="30d24fbb3de682424e95a56fad4946c155d3eb9c6f66876f52c42d1f87226a5b"
Jul 17 18:20:27.958221 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:27.957940    2572 scope.go:117] "RemoveContainer" containerID="f29f9fb560f9b5d9f9f26a0c2712811388f2fe2379a179cc5236e356a0a4d009"
Jul 17 18:20:27.958221 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:27.958177    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q_odh-ai-gateway-infra(a7947789-0533-4755-beb6-3b5e7b58ad00)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00"
Jul 17 18:20:28.965663 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:28.965633    2572 scope.go:117] "RemoveContainer" containerID="f29f9fb560f9b5d9f9f26a0c2712811388f2fe2379a179cc5236e356a0a4d009"
Jul 17 18:20:28.966078 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:28.965806    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q_odh-ai-gateway-infra(a7947789-0533-4755-beb6-3b5e7b58ad00)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00"
Jul 17 18:20:32.571048 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:32.571010    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"]
Jul 17 18:20:32.574352 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:32.574330    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"
Jul 17 18:20:32.583530 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:32.583323    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"]
Jul 17 18:20:32.586403 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:32.586365    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b294fe6-3766-4604-9d48-d25322f79c70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v\" (UID: \"8b294fe6-3766-4604-9d48-d25322f79c70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"
Jul 17 18:20:32.687360 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:32.687321    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b294fe6-3766-4604-9d48-d25322f79c70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v\" (UID: \"8b294fe6-3766-4604-9d48-d25322f79c70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"
Jul 17 18:20:32.688009 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:32.687989    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b294fe6-3766-4604-9d48-d25322f79c70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v\" (UID: \"8b294fe6-3766-4604-9d48-d25322f79c70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"
Jul 17 18:20:32.887305 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:32.887269    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"
Jul 17 18:20:33.039888 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:33.039850    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"]
Jul 17 18:20:33.041729 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:20:33.041693    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b294fe6_3766_4604_9d48_d25322f79c70.slice/crio-4c157eadb119d3a732944c715f12b69213d9f684392db9d93fc86d0fd8477778 WatchSource:0}: Error finding container 4c157eadb119d3a732944c715f12b69213d9f684392db9d93fc86d0fd8477778: Status 404 returned error can't find the container with id 4c157eadb119d3a732944c715f12b69213d9f684392db9d93fc86d0fd8477778
Jul 17 18:20:33.998898 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:33.998862    2572 generic.go:356] "Generic (PLEG): container finished" podID="8b294fe6-3766-4604-9d48-d25322f79c70" containerID="f8838cdbeb7cb3b76708816d6f9f9282b2d4da7938726c87905e0ae428eef2f3" exitCode=0
Jul 17 18:20:33.999333 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:33.998956    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v" event={"ID":"8b294fe6-3766-4604-9d48-d25322f79c70","Type":"ContainerDied","Data":"f8838cdbeb7cb3b76708816d6f9f9282b2d4da7938726c87905e0ae428eef2f3"}
Jul 17 18:20:33.999333 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:33.999023    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v" event={"ID":"8b294fe6-3766-4604-9d48-d25322f79c70","Type":"ContainerStarted","Data":"4c157eadb119d3a732944c715f12b69213d9f684392db9d93fc86d0fd8477778"}
Jul 17 18:20:34.509965 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:34.509923    2572 scope.go:117] "RemoveContainer" containerID="2ae637d0f34664c7b606669d0c6bb1b1259270868de0b7232bc212abc964e466"
Jul 17 18:20:35.014095 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:35.014055    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" event={"ID":"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a","Type":"ContainerStarted","Data":"94cb0bf90afd7fceb2c13e8c6fc6963abd2503364e7be3a81232cfcc539f3371"}
Jul 17 18:20:35.032791 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:35.032731    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" podStartSLOduration=18.032710622 podStartE2EDuration="18.032710622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:20:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:20:35.029535593 +0000 UTC m=+3407.034891448" watchObservedRunningTime="2026-07-17 18:20:35.032710622 +0000 UTC m=+3407.038066539"
Jul 17 18:20:35.198261 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:35.198226    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"
Jul 17 18:20:35.315952 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:35.315921    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b294fe6-3766-4604-9d48-d25322f79c70-maas-api-service-ca\") pod \"8b294fe6-3766-4604-9d48-d25322f79c70\" (UID: \"8b294fe6-3766-4604-9d48-d25322f79c70\") "
Jul 17 18:20:35.316330 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:35.316305    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b294fe6-3766-4604-9d48-d25322f79c70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8b294fe6-3766-4604-9d48-d25322f79c70" (UID: "8b294fe6-3766-4604-9d48-d25322f79c70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:20:35.416735 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:35.416694    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b294fe6-3766-4604-9d48-d25322f79c70-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:20:36.020938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:36.020908    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v"
Jul 17 18:20:36.021372 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:36.020946    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v" event={"ID":"8b294fe6-3766-4604-9d48-d25322f79c70","Type":"ContainerDied","Data":"4c157eadb119d3a732944c715f12b69213d9f684392db9d93fc86d0fd8477778"}
Jul 17 18:20:36.021372 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:36.020997    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c157eadb119d3a732944c715f12b69213d9f684392db9d93fc86d0fd8477778"
Jul 17 18:20:36.022949 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:36.022922    2572 generic.go:356] "Generic (PLEG): container finished" podID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerID="94cb0bf90afd7fceb2c13e8c6fc6963abd2503364e7be3a81232cfcc539f3371" exitCode=6
Jul 17 18:20:36.023082 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:36.022990    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" event={"ID":"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a","Type":"ContainerDied","Data":"94cb0bf90afd7fceb2c13e8c6fc6963abd2503364e7be3a81232cfcc539f3371"}
Jul 17 18:20:36.023082 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:36.023038    2572 scope.go:117] "RemoveContainer" containerID="2ae637d0f34664c7b606669d0c6bb1b1259270868de0b7232bc212abc964e466"
Jul 17 18:20:36.023331 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:36.023305    2572 scope.go:117] "RemoveContainer" containerID="94cb0bf90afd7fceb2c13e8c6fc6963abd2503364e7be3a81232cfcc539f3371"
Jul 17 18:20:36.023542 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:36.023521    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c_odh-ai-gateway-infra(4aafc40a-e7b2-4db5-b0c3-2d9adf47618a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a"
Jul 17 18:20:36.040634 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:36.040606    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"]
Jul 17 18:20:36.510425 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:36.510392    2572 scope.go:117] "RemoveContainer" containerID="d3a7daab2bdfa57d8bc4ca350641e5f06c735e0d116d08188b659937419a805a"
Jul 17 18:20:37.032798 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:37.032765    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" event={"ID":"e0844fe0-fdb2-4063-929d-698cce105dcd","Type":"ContainerStarted","Data":"489a173feb1ad034438765a9e4a1d4ac833ef64f640722161af8d525797e028b"}
Jul 17 18:20:37.172282 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:37.172255    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"
Jul 17 18:20:37.231894 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:37.231865    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aafc40a-e7b2-4db5-b0c3-2d9adf47618a-maas-api-service-ca\") pod \"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a\" (UID: \"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a\") "
Jul 17 18:20:37.232246 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:37.232222    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aafc40a-e7b2-4db5-b0c3-2d9adf47618a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" (UID: "4aafc40a-e7b2-4db5-b0c3-2d9adf47618a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:20:37.332470 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:37.332427    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aafc40a-e7b2-4db5-b0c3-2d9adf47618a-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:20:38.039797 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.039756    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c" event={"ID":"4aafc40a-e7b2-4db5-b0c3-2d9adf47618a","Type":"ContainerDied","Data":"c00811cfa15b314148353a4fdad8b9771889e567e63e0fe705adbec07db2247a"}
Jul 17 18:20:38.039797 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.039731    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"
Jul 17 18:20:38.040350 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.039841    2572 scope.go:117] "RemoveContainer" containerID="94cb0bf90afd7fceb2c13e8c6fc6963abd2503364e7be3a81232cfcc539f3371"
Jul 17 18:20:38.045610 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.045578    2572 generic.go:356] "Generic (PLEG): container finished" podID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerID="489a173feb1ad034438765a9e4a1d4ac833ef64f640722161af8d525797e028b" exitCode=6
Jul 17 18:20:38.045789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.045611    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" event={"ID":"e0844fe0-fdb2-4063-929d-698cce105dcd","Type":"ContainerDied","Data":"489a173feb1ad034438765a9e4a1d4ac833ef64f640722161af8d525797e028b"}
Jul 17 18:20:38.045953 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.045936    2572 scope.go:117] "RemoveContainer" containerID="489a173feb1ad034438765a9e4a1d4ac833ef64f640722161af8d525797e028b"
Jul 17 18:20:38.046177 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:38.046160    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl_odh-ai-gateway-infra(e0844fe0-fdb2-4063-929d-698cce105dcd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd"
Jul 17 18:20:38.053247 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.053229    2572 scope.go:117] "RemoveContainer" containerID="d3a7daab2bdfa57d8bc4ca350641e5f06c735e0d116d08188b659937419a805a"
Jul 17 18:20:38.073159 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.073127    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"]
Jul 17 18:20:38.078910 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.078880    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"]
Jul 17 18:20:38.082576 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.082548    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-ngm2c"]
Jul 17 18:20:38.523368 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:38.523331    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" path="/var/lib/kubelet/pods/4aafc40a-e7b2-4db5-b0c3-2d9adf47618a/volumes"
Jul 17 18:20:39.195584 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:39.195552    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"
Jul 17 18:20:39.248319 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:39.248279    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0844fe0-fdb2-4063-929d-698cce105dcd-maas-api-service-ca\") pod \"e0844fe0-fdb2-4063-929d-698cce105dcd\" (UID: \"e0844fe0-fdb2-4063-929d-698cce105dcd\") "
Jul 17 18:20:39.248692 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:39.248669    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0844fe0-fdb2-4063-929d-698cce105dcd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e0844fe0-fdb2-4063-929d-698cce105dcd" (UID: "e0844fe0-fdb2-4063-929d-698cce105dcd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:20:39.349792 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:39.349756    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0844fe0-fdb2-4063-929d-698cce105dcd-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:20:40.060837 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:40.060802    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"
Jul 17 18:20:40.060837 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:40.060814    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl" event={"ID":"e0844fe0-fdb2-4063-929d-698cce105dcd","Type":"ContainerDied","Data":"eef55f66e3129ae572fc334d7d8450e846fc434becb88bed3da60415ebcfb09b"}
Jul 17 18:20:40.061116 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:40.060886    2572 scope.go:117] "RemoveContainer" containerID="489a173feb1ad034438765a9e4a1d4ac833ef64f640722161af8d525797e028b"
Jul 17 18:20:40.087955 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:40.087913    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"]
Jul 17 18:20:40.093563 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:40.093526    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-l2ldl"]
Jul 17 18:20:40.516177 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:40.516039    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" path="/var/lib/kubelet/pods/e0844fe0-fdb2-4063-929d-698cce105dcd/volumes"
Jul 17 18:20:43.012549 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:43.012503    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p"]
Jul 17 18:20:43.018215 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:43.018180    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4699ff-29738535-l4t8p"]
Jul 17 18:20:43.509948 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:43.509910    2572 scope.go:117] "RemoveContainer" containerID="f29f9fb560f9b5d9f9f26a0c2712811388f2fe2379a179cc5236e356a0a4d009"
Jul 17 18:20:44.088042 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:44.088001    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" event={"ID":"a7947789-0533-4755-beb6-3b5e7b58ad00","Type":"ContainerStarted","Data":"da70801564c13f82e31abe980c9e835130a0b66a8dd6547e96f59066914fd560"}
Jul 17 18:20:44.516017 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:44.515931    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffeeee75-61c9-46fc-9cc3-353864bc8fb0" path="/var/lib/kubelet/pods/ffeeee75-61c9-46fc-9cc3-353864bc8fb0/volumes"
Jul 17 18:20:45.095903 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:45.095869    2572 generic.go:356] "Generic (PLEG): container finished" podID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerID="da70801564c13f82e31abe980c9e835130a0b66a8dd6547e96f59066914fd560" exitCode=6
Jul 17 18:20:45.096319 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:45.095942    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" event={"ID":"a7947789-0533-4755-beb6-3b5e7b58ad00","Type":"ContainerDied","Data":"da70801564c13f82e31abe980c9e835130a0b66a8dd6547e96f59066914fd560"}
Jul 17 18:20:45.096319 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:45.096005    2572 scope.go:117] "RemoveContainer" containerID="f29f9fb560f9b5d9f9f26a0c2712811388f2fe2379a179cc5236e356a0a4d009"
Jul 17 18:20:45.096319 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:45.096296    2572 scope.go:117] "RemoveContainer" containerID="da70801564c13f82e31abe980c9e835130a0b66a8dd6547e96f59066914fd560"
Jul 17 18:20:45.096542 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:45.096525    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q_odh-ai-gateway-infra(a7947789-0533-4755-beb6-3b5e7b58ad00)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00"
Jul 17 18:20:45.119838 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:45.119807    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"]
Jul 17 18:20:46.251247 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:46.251219    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"
Jul 17 18:20:46.310181 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:46.310137    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7947789-0533-4755-beb6-3b5e7b58ad00-maas-api-service-ca\") pod \"a7947789-0533-4755-beb6-3b5e7b58ad00\" (UID: \"a7947789-0533-4755-beb6-3b5e7b58ad00\") "
Jul 17 18:20:46.310543 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:46.310484    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7947789-0533-4755-beb6-3b5e7b58ad00-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7947789-0533-4755-beb6-3b5e7b58ad00" (UID: "a7947789-0533-4755-beb6-3b5e7b58ad00"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:20:46.410918 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:46.410811    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7947789-0533-4755-beb6-3b5e7b58ad00-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:20:47.114959 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:47.114929    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"
Jul 17 18:20:47.114959 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:47.114938    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q" event={"ID":"a7947789-0533-4755-beb6-3b5e7b58ad00","Type":"ContainerDied","Data":"32cbff7604a6af19ea655459dd19826b28918a242a90da8e7070ade1e149cfe2"}
Jul 17 18:20:47.115230 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:47.114994    2572 scope.go:117] "RemoveContainer" containerID="da70801564c13f82e31abe980c9e835130a0b66a8dd6547e96f59066914fd560"
Jul 17 18:20:47.153812 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:47.153770    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"]
Jul 17 18:20:47.163336 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:47.163307    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-l7m2q"]
Jul 17 18:20:48.516943 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:48.516907    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" path="/var/lib/kubelet/pods/a7947789-0533-4755-beb6-3b5e7b58ad00/volumes"
Jul 17 18:20:50.724761 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.724712    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"]
Jul 17 18:20:50.725228 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725210    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerName="revoke-keys"
Jul 17 18:20:50.725276 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725231    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerName="revoke-keys"
Jul 17 18:20:50.725276 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725244    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerName="revoke-keys"
Jul 17 18:20:50.725276 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725253    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerName="revoke-keys"
Jul 17 18:20:50.725276 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725263    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerName="revoke-keys"
Jul 17 18:20:50.725276 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725273    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725285    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725294    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725302    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b294fe6-3766-4604-9d48-d25322f79c70" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725310    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b294fe6-3766-4604-9d48-d25322f79c70" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725332    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725342    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725353    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725362    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725378    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerName="revoke-keys"
Jul 17 18:20:50.725426 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725386    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerName="revoke-keys"
Jul 17 18:20:50.725755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725465    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b294fe6-3766-4604-9d48-d25322f79c70" containerName="revoke-keys"
Jul 17 18:20:50.725755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725479    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerName="revoke-keys"
Jul 17 18:20:50.725755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725505    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerName="revoke-keys"
Jul 17 18:20:50.725755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725514    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerName="revoke-keys"
Jul 17 18:20:50.725755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725524    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerName="revoke-keys"
Jul 17 18:20:50.725755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725533    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerName="revoke-keys"
Jul 17 18:20:50.725755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725542    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aafc40a-e7b2-4db5-b0c3-2d9adf47618a" containerName="revoke-keys"
Jul 17 18:20:50.725755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725552    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerName="revoke-keys"
Jul 17 18:20:50.725755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.725564    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerName="revoke-keys"
Jul 17 18:20:50.730932 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.730905    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"
Jul 17 18:20:50.734054 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.734024    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:20:50.734202 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.734027    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:20:50.734455 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.734430    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"]
Jul 17 18:20:50.850993 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.850956    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87dcc8c-b962-414a-b920-4f5b7265dfac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c\" (UID: \"a87dcc8c-b962-414a-b920-4f5b7265dfac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"
Jul 17 18:20:50.952139 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.952097    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87dcc8c-b962-414a-b920-4f5b7265dfac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c\" (UID: \"a87dcc8c-b962-414a-b920-4f5b7265dfac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"
Jul 17 18:20:50.952818 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:50.952790    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87dcc8c-b962-414a-b920-4f5b7265dfac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c\" (UID: \"a87dcc8c-b962-414a-b920-4f5b7265dfac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"
Jul 17 18:20:51.044481 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:51.044388    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"
Jul 17 18:20:51.190583 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:51.190554    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"]
Jul 17 18:20:51.192304 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:20:51.192275    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda87dcc8c_b962_414a_b920_4f5b7265dfac.slice/crio-643a712285e8775619cd3a7630d0af1004755c6656cd1c3cb3acf04df9d58678 WatchSource:0}: Error finding container 643a712285e8775619cd3a7630d0af1004755c6656cd1c3cb3acf04df9d58678: Status 404 returned error can't find the container with id 643a712285e8775619cd3a7630d0af1004755c6656cd1c3cb3acf04df9d58678
Jul 17 18:20:52.147676 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:52.147576    2572 generic.go:356] "Generic (PLEG): container finished" podID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerID="896cce15459bd79da00372a028d6b720586e29d4aa2ea00644a9771a6a1e61c1" exitCode=6
Jul 17 18:20:52.147676 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:52.147665    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c" event={"ID":"a87dcc8c-b962-414a-b920-4f5b7265dfac","Type":"ContainerDied","Data":"896cce15459bd79da00372a028d6b720586e29d4aa2ea00644a9771a6a1e61c1"}
Jul 17 18:20:52.148110 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:52.147723    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c" event={"ID":"a87dcc8c-b962-414a-b920-4f5b7265dfac","Type":"ContainerStarted","Data":"643a712285e8775619cd3a7630d0af1004755c6656cd1c3cb3acf04df9d58678"}
Jul 17 18:20:52.148110 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:52.147835    2572 scope.go:117] "RemoveContainer" containerID="896cce15459bd79da00372a028d6b720586e29d4aa2ea00644a9771a6a1e61c1"
Jul 17 18:20:53.155687 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:53.155642    2572 generic.go:356] "Generic (PLEG): container finished" podID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerID="ec3d631645664528eccbf74652745967bf79350b5623e01ca66ad267099f1bc1" exitCode=6
Jul 17 18:20:53.156126 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:53.155730    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c" event={"ID":"a87dcc8c-b962-414a-b920-4f5b7265dfac","Type":"ContainerDied","Data":"ec3d631645664528eccbf74652745967bf79350b5623e01ca66ad267099f1bc1"}
Jul 17 18:20:53.156126 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:53.155798    2572 scope.go:117] "RemoveContainer" containerID="896cce15459bd79da00372a028d6b720586e29d4aa2ea00644a9771a6a1e61c1"
Jul 17 18:20:53.156126 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:53.156037    2572 scope.go:117] "RemoveContainer" containerID="ec3d631645664528eccbf74652745967bf79350b5623e01ca66ad267099f1bc1"
Jul 17 18:20:53.156306 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:53.156286    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c_odh-ai-gateway-infra(a87dcc8c-b962-414a-b920-4f5b7265dfac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac"
Jul 17 18:20:54.163987 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:54.163949    2572 scope.go:117] "RemoveContainer" containerID="ec3d631645664528eccbf74652745967bf79350b5623e01ca66ad267099f1bc1"
Jul 17 18:20:54.164392 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:20:54.164166    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c_odh-ai-gateway-infra(a87dcc8c-b962-414a-b920-4f5b7265dfac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac"
Jul 17 18:20:58.721447 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.721401    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"]
Jul 17 18:20:58.721915 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.721781    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerName="revoke-keys"
Jul 17 18:20:58.721915 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.721794    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7947789-0533-4755-beb6-3b5e7b58ad00" containerName="revoke-keys"
Jul 17 18:20:58.721915 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.721813    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerName="revoke-keys"
Jul 17 18:20:58.721915 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.721818    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerName="revoke-keys"
Jul 17 18:20:58.721915 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.721880    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0844fe0-fdb2-4063-929d-698cce105dcd" containerName="revoke-keys"
Jul 17 18:20:58.726126 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.726104    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"
Jul 17 18:20:58.732872 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.732839    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"]
Jul 17 18:20:58.824100 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.824061    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49cdf2b5-e249-4675-b8ad-8933b6d65d41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q\" (UID: \"49cdf2b5-e249-4675-b8ad-8933b6d65d41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"
Jul 17 18:20:58.925550 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.925502    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49cdf2b5-e249-4675-b8ad-8933b6d65d41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q\" (UID: \"49cdf2b5-e249-4675-b8ad-8933b6d65d41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"
Jul 17 18:20:58.926165 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:58.926107    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49cdf2b5-e249-4675-b8ad-8933b6d65d41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q\" (UID: \"49cdf2b5-e249-4675-b8ad-8933b6d65d41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"
Jul 17 18:20:59.039262 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:59.039153    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"
Jul 17 18:20:59.189358 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:20:59.189330    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"]
Jul 17 18:20:59.191742 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:20:59.191701    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49cdf2b5_e249_4675_b8ad_8933b6d65d41.slice/crio-58b3d677fcad358ba05a0aeb1f33dce88b7a7ddb00bc4f6a555fe8c7c8fda159 WatchSource:0}: Error finding container 58b3d677fcad358ba05a0aeb1f33dce88b7a7ddb00bc4f6a555fe8c7c8fda159: Status 404 returned error can't find the container with id 58b3d677fcad358ba05a0aeb1f33dce88b7a7ddb00bc4f6a555fe8c7c8fda159
Jul 17 18:21:00.201398 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:00.201364    2572 generic.go:356] "Generic (PLEG): container finished" podID="49cdf2b5-e249-4675-b8ad-8933b6d65d41" containerID="b77428d65bf31791cba98bbfdbb223eee97556e9c17912380b44741d74ab7946" exitCode=0
Jul 17 18:21:00.201851 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:00.201459    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q" event={"ID":"49cdf2b5-e249-4675-b8ad-8933b6d65d41","Type":"ContainerDied","Data":"b77428d65bf31791cba98bbfdbb223eee97556e9c17912380b44741d74ab7946"}
Jul 17 18:21:00.201851 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:00.201534    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q" event={"ID":"49cdf2b5-e249-4675-b8ad-8933b6d65d41","Type":"ContainerStarted","Data":"58b3d677fcad358ba05a0aeb1f33dce88b7a7ddb00bc4f6a555fe8c7c8fda159"}
Jul 17 18:21:01.354187 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:01.354160    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"
Jul 17 18:21:01.447816 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:01.447777    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49cdf2b5-e249-4675-b8ad-8933b6d65d41-maas-api-service-ca\") pod \"49cdf2b5-e249-4675-b8ad-8933b6d65d41\" (UID: \"49cdf2b5-e249-4675-b8ad-8933b6d65d41\") "
Jul 17 18:21:01.448188 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:01.448164    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49cdf2b5-e249-4675-b8ad-8933b6d65d41-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "49cdf2b5-e249-4675-b8ad-8933b6d65d41" (UID: "49cdf2b5-e249-4675-b8ad-8933b6d65d41"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:21:01.549272 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:01.549177    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49cdf2b5-e249-4675-b8ad-8933b6d65d41-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:21:02.213864 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:02.213830    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q"
Jul 17 18:21:02.213864 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:02.213857    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q" event={"ID":"49cdf2b5-e249-4675-b8ad-8933b6d65d41","Type":"ContainerDied","Data":"58b3d677fcad358ba05a0aeb1f33dce88b7a7ddb00bc4f6a555fe8c7c8fda159"}
Jul 17 18:21:02.214077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:02.213931    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58b3d677fcad358ba05a0aeb1f33dce88b7a7ddb00bc4f6a555fe8c7c8fda159"
Jul 17 18:21:05.509943 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:05.509910    2572 scope.go:117] "RemoveContainer" containerID="ec3d631645664528eccbf74652745967bf79350b5623e01ca66ad267099f1bc1"
Jul 17 18:21:06.241061 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:06.241026    2572 generic.go:356] "Generic (PLEG): container finished" podID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerID="6c7422fe70dabe1df91f2d2a94220625dc606dd1cbf8a5823d8f07f83c330c4f" exitCode=6
Jul 17 18:21:06.241238 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:06.241096    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c" event={"ID":"a87dcc8c-b962-414a-b920-4f5b7265dfac","Type":"ContainerDied","Data":"6c7422fe70dabe1df91f2d2a94220625dc606dd1cbf8a5823d8f07f83c330c4f"}
Jul 17 18:21:06.241238 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:06.241155    2572 scope.go:117] "RemoveContainer" containerID="ec3d631645664528eccbf74652745967bf79350b5623e01ca66ad267099f1bc1"
Jul 17 18:21:06.241449 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:06.241420    2572 scope.go:117] "RemoveContainer" containerID="6c7422fe70dabe1df91f2d2a94220625dc606dd1cbf8a5823d8f07f83c330c4f"
Jul 17 18:21:06.241699 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:06.241675    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c_odh-ai-gateway-infra(a87dcc8c-b962-414a-b920-4f5b7265dfac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac"
Jul 17 18:21:07.267598 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:07.267557    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"]
Jul 17 18:21:07.409053 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:07.409028    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"
Jul 17 18:21:07.507655 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:07.507620    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87dcc8c-b962-414a-b920-4f5b7265dfac-maas-api-service-ca\") pod \"a87dcc8c-b962-414a-b920-4f5b7265dfac\" (UID: \"a87dcc8c-b962-414a-b920-4f5b7265dfac\") "
Jul 17 18:21:07.508008 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:07.507982    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a87dcc8c-b962-414a-b920-4f5b7265dfac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a87dcc8c-b962-414a-b920-4f5b7265dfac" (UID: "a87dcc8c-b962-414a-b920-4f5b7265dfac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:21:07.608761 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:07.608720    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87dcc8c-b962-414a-b920-4f5b7265dfac-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:21:08.255643 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:08.255600    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c" event={"ID":"a87dcc8c-b962-414a-b920-4f5b7265dfac","Type":"ContainerDied","Data":"643a712285e8775619cd3a7630d0af1004755c6656cd1c3cb3acf04df9d58678"}
Jul 17 18:21:08.255838 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:08.255672    2572 scope.go:117] "RemoveContainer" containerID="6c7422fe70dabe1df91f2d2a94220625dc606dd1cbf8a5823d8f07f83c330c4f"
Jul 17 18:21:08.255838 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:08.255615    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"
Jul 17 18:21:08.280118 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:08.280082    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"]
Jul 17 18:21:08.285679 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:08.285640    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-wdj4c"]
Jul 17 18:21:08.515277 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:08.515192    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" path="/var/lib/kubelet/pods/a87dcc8c-b962-414a-b920-4f5b7265dfac/volumes"
Jul 17 18:21:09.258709 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:09.258678    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59"]
Jul 17 18:21:09.261266 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:09.261239    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-22fb10-29738535-zbh59"]
Jul 17 18:21:10.516431 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:10.516391    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06a94db1-06f4-435e-ae49-38df53761038" path="/var/lib/kubelet/pods/06a94db1-06f4-435e-ae49-38df53761038/volumes"
Jul 17 18:21:17.227131 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227093    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"]
Jul 17 18:21:17.227558 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227433    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerName="revoke-keys"
Jul 17 18:21:17.227558 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227445    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerName="revoke-keys"
Jul 17 18:21:17.227558 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227460    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerName="revoke-keys"
Jul 17 18:21:17.227558 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227466    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerName="revoke-keys"
Jul 17 18:21:17.227558 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227480    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49cdf2b5-e249-4675-b8ad-8933b6d65d41" containerName="revoke-keys"
Jul 17 18:21:17.227558 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227485    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="49cdf2b5-e249-4675-b8ad-8933b6d65d41" containerName="revoke-keys"
Jul 17 18:21:17.227558 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227553    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerName="revoke-keys"
Jul 17 18:21:17.227558 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227564    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="49cdf2b5-e249-4675-b8ad-8933b6d65d41" containerName="revoke-keys"
Jul 17 18:21:17.227817 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.227571    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerName="revoke-keys"
Jul 17 18:21:17.232106 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.232085    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"
Jul 17 18:21:17.234938 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.234899    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x4f9n\""
Jul 17 18:21:17.235101 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.234946    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:21:17.239725 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.239697    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"]
Jul 17 18:21:17.395016 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.394962    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-97vft\" (UID: \"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"
Jul 17 18:21:17.495949 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.495833    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-97vft\" (UID: \"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"
Jul 17 18:21:17.496509 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.496471    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-59516a0f-97vft\" (UID: \"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"
Jul 17 18:21:17.545096 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.545055    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"
Jul 17 18:21:17.705349 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:17.705317    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"]
Jul 17 18:21:17.706574 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:21:17.706544    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8ea6c7_97aa_44d4_8cae_fa7a6e31cafc.slice/crio-bc6766c94650531c7ad676f537a53b765d945b6734761463d39905f98d96e0a0 WatchSource:0}: Error finding container bc6766c94650531c7ad676f537a53b765d945b6734761463d39905f98d96e0a0: Status 404 returned error can't find the container with id bc6766c94650531c7ad676f537a53b765d945b6734761463d39905f98d96e0a0
Jul 17 18:21:18.324281 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:18.324235    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" event={"ID":"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc","Type":"ContainerStarted","Data":"8d8b012f713eb4024398dab326dfc735afa66d5331172afda99f83b4e18c043f"}
Jul 17 18:21:18.324759 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:18.324331    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" event={"ID":"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc","Type":"ContainerStarted","Data":"bc6766c94650531c7ad676f537a53b765d945b6734761463d39905f98d96e0a0"}
Jul 17 18:21:18.340959 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:18.340900    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" podStartSLOduration=1.340879313 podStartE2EDuration="1.340879313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:21:18.339801043 +0000 UTC m=+3450.345156925" watchObservedRunningTime="2026-07-17 18:21:18.340879313 +0000 UTC m=+3450.346235173"
Jul 17 18:21:19.112032 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.111989    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"]
Jul 17 18:21:19.112441 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.112425    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerName="revoke-keys"
Jul 17 18:21:19.112511 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.112443    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerName="revoke-keys"
Jul 17 18:21:19.112553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.112521    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87dcc8c-b962-414a-b920-4f5b7265dfac" containerName="revoke-keys"
Jul 17 18:21:19.115740 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.115721    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"
Jul 17 18:21:19.125597 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.125568    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"]
Jul 17 18:21:19.209292 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.209249    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a41ada-2fad-41c3-a449-99c7807099cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf\" (UID: \"95a41ada-2fad-41c3-a449-99c7807099cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"
Jul 17 18:21:19.310738 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.310697    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a41ada-2fad-41c3-a449-99c7807099cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf\" (UID: \"95a41ada-2fad-41c3-a449-99c7807099cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"
Jul 17 18:21:19.311374 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.311347    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a41ada-2fad-41c3-a449-99c7807099cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf\" (UID: \"95a41ada-2fad-41c3-a449-99c7807099cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"
Jul 17 18:21:19.332297 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.332255    2572 generic.go:356] "Generic (PLEG): container finished" podID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerID="8d8b012f713eb4024398dab326dfc735afa66d5331172afda99f83b4e18c043f" exitCode=6
Jul 17 18:21:19.332790 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.332347    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" event={"ID":"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc","Type":"ContainerDied","Data":"8d8b012f713eb4024398dab326dfc735afa66d5331172afda99f83b4e18c043f"}
Jul 17 18:21:19.332790 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.332526    2572 scope.go:117] "RemoveContainer" containerID="8d8b012f713eb4024398dab326dfc735afa66d5331172afda99f83b4e18c043f"
Jul 17 18:21:19.427693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.427655    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"
Jul 17 18:21:19.578114 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:19.578083    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"]
Jul 17 18:21:19.579810 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:21:19.579772    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a41ada_2fad_41c3_a449_99c7807099cb.slice/crio-0f890a00c0859c76c567930509f85a884f6a270bf87eaf7078a82502844babfb WatchSource:0}: Error finding container 0f890a00c0859c76c567930509f85a884f6a270bf87eaf7078a82502844babfb: Status 404 returned error can't find the container with id 0f890a00c0859c76c567930509f85a884f6a270bf87eaf7078a82502844babfb
Jul 17 18:21:20.340758 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:20.340719    2572 generic.go:356] "Generic (PLEG): container finished" podID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerID="4127ca7826c8562c69c56dbd430fe5683e1ab4f05dd3b3bfb13da626de2809ac" exitCode=6
Jul 17 18:21:20.341219 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:20.340801    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" event={"ID":"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc","Type":"ContainerDied","Data":"4127ca7826c8562c69c56dbd430fe5683e1ab4f05dd3b3bfb13da626de2809ac"}
Jul 17 18:21:20.341219 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:20.340860    2572 scope.go:117] "RemoveContainer" containerID="8d8b012f713eb4024398dab326dfc735afa66d5331172afda99f83b4e18c043f"
Jul 17 18:21:20.341219 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:20.341050    2572 scope.go:117] "RemoveContainer" containerID="4127ca7826c8562c69c56dbd430fe5683e1ab4f05dd3b3bfb13da626de2809ac"
Jul 17 18:21:20.341410 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:20.341290    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-97vft_odh-ai-gateway-infra(8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc"
Jul 17 18:21:20.342446 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:20.342420    2572 generic.go:356] "Generic (PLEG): container finished" podID="95a41ada-2fad-41c3-a449-99c7807099cb" containerID="0387e8527ef8a045623f0ea2c3aa777847d13fbabca94b5901d5368ee1d90950" exitCode=6
Jul 17 18:21:20.342593 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:20.342533    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf" event={"ID":"95a41ada-2fad-41c3-a449-99c7807099cb","Type":"ContainerDied","Data":"0387e8527ef8a045623f0ea2c3aa777847d13fbabca94b5901d5368ee1d90950"}
Jul 17 18:21:20.342641 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:20.342596    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf" event={"ID":"95a41ada-2fad-41c3-a449-99c7807099cb","Type":"ContainerStarted","Data":"0f890a00c0859c76c567930509f85a884f6a270bf87eaf7078a82502844babfb"}
Jul 17 18:21:20.342774 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:20.342761    2572 scope.go:117] "RemoveContainer" containerID="0387e8527ef8a045623f0ea2c3aa777847d13fbabca94b5901d5368ee1d90950"
Jul 17 18:21:21.350789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:21.350756    2572 scope.go:117] "RemoveContainer" containerID="4127ca7826c8562c69c56dbd430fe5683e1ab4f05dd3b3bfb13da626de2809ac"
Jul 17 18:21:21.351302 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:21.351022    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-97vft_odh-ai-gateway-infra(8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc"
Jul 17 18:21:21.352593 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:21.352569    2572 generic.go:356] "Generic (PLEG): container finished" podID="95a41ada-2fad-41c3-a449-99c7807099cb" containerID="90470de2f5f5b1fdb8fe0d0fbbde1408b10b41d54da90a8e4bbd981d716635f7" exitCode=6
Jul 17 18:21:21.352700 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:21.352677    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf" event={"ID":"95a41ada-2fad-41c3-a449-99c7807099cb","Type":"ContainerDied","Data":"90470de2f5f5b1fdb8fe0d0fbbde1408b10b41d54da90a8e4bbd981d716635f7"}
Jul 17 18:21:21.352778 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:21.352749    2572 scope.go:117] "RemoveContainer" containerID="0387e8527ef8a045623f0ea2c3aa777847d13fbabca94b5901d5368ee1d90950"
Jul 17 18:21:21.352845 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:21.352832    2572 scope.go:117] "RemoveContainer" containerID="90470de2f5f5b1fdb8fe0d0fbbde1408b10b41d54da90a8e4bbd981d716635f7"
Jul 17 18:21:21.353034 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:21.353012    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf_odh-ai-gateway-infra(95a41ada-2fad-41c3-a449-99c7807099cb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf" podUID="95a41ada-2fad-41c3-a449-99c7807099cb"
Jul 17 18:21:22.360755 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:22.360729    2572 scope.go:117] "RemoveContainer" containerID="90470de2f5f5b1fdb8fe0d0fbbde1408b10b41d54da90a8e4bbd981d716635f7"
Jul 17 18:21:22.361196 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:22.360991    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf_odh-ai-gateway-infra(95a41ada-2fad-41c3-a449-99c7807099cb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf" podUID="95a41ada-2fad-41c3-a449-99c7807099cb"
Jul 17 18:21:24.992885 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:24.992841    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"]
Jul 17 18:21:24.997693 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:24.997663    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"
Jul 17 18:21:25.004122 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:25.004079    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"]
Jul 17 18:21:25.065217 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:25.065173    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c277a4-b8bb-4689-bd0d-c5edec30e1f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j\" (UID: \"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"
Jul 17 18:21:25.166657 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:25.166613    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c277a4-b8bb-4689-bd0d-c5edec30e1f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j\" (UID: \"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"
Jul 17 18:21:25.167332 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:25.167301    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c277a4-b8bb-4689-bd0d-c5edec30e1f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j\" (UID: \"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"
Jul 17 18:21:25.310799 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:25.310759    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"
Jul 17 18:21:25.456068 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:25.456039    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"]
Jul 17 18:21:25.458313 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:21:25.458282    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c277a4_b8bb_4689_bd0d_c5edec30e1f0.slice/crio-e634ebc145dd0953120e2cc3cdc2ede9fca1b41eb41f08d4f72391d22a7dd8b7 WatchSource:0}: Error finding container e634ebc145dd0953120e2cc3cdc2ede9fca1b41eb41f08d4f72391d22a7dd8b7: Status 404 returned error can't find the container with id e634ebc145dd0953120e2cc3cdc2ede9fca1b41eb41f08d4f72391d22a7dd8b7
Jul 17 18:21:26.391389 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:26.391291    2572 generic.go:356] "Generic (PLEG): container finished" podID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" containerID="dd0e25d249e334f21822d51cce484f52ea997a74cfda391dbcbf05c5dea9f55c" exitCode=6
Jul 17 18:21:26.391780 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:26.391385    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j" event={"ID":"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0","Type":"ContainerDied","Data":"dd0e25d249e334f21822d51cce484f52ea997a74cfda391dbcbf05c5dea9f55c"}
Jul 17 18:21:26.391780 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:26.391444    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j" event={"ID":"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0","Type":"ContainerStarted","Data":"e634ebc145dd0953120e2cc3cdc2ede9fca1b41eb41f08d4f72391d22a7dd8b7"}
Jul 17 18:21:26.391780 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:26.391582    2572 scope.go:117] "RemoveContainer" containerID="dd0e25d249e334f21822d51cce484f52ea997a74cfda391dbcbf05c5dea9f55c"
Jul 17 18:21:27.400080 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:27.400047    2572 generic.go:356] "Generic (PLEG): container finished" podID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" containerID="f1b4db3871182fed1a845651f6cdd77ec8e17ef71bf74a85792ed9ac29638d43" exitCode=6
Jul 17 18:21:27.400553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:27.400139    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j" event={"ID":"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0","Type":"ContainerDied","Data":"f1b4db3871182fed1a845651f6cdd77ec8e17ef71bf74a85792ed9ac29638d43"}
Jul 17 18:21:27.400553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:27.400201    2572 scope.go:117] "RemoveContainer" containerID="dd0e25d249e334f21822d51cce484f52ea997a74cfda391dbcbf05c5dea9f55c"
Jul 17 18:21:27.400553 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:27.400417    2572 scope.go:117] "RemoveContainer" containerID="f1b4db3871182fed1a845651f6cdd77ec8e17ef71bf74a85792ed9ac29638d43"
Jul 17 18:21:27.400770 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:27.400696    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j_odh-ai-gateway-infra(f1c277a4-b8bb-4689-bd0d-c5edec30e1f0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0"
Jul 17 18:21:28.407643 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:28.407615    2572 scope.go:117] "RemoveContainer" containerID="f1b4db3871182fed1a845651f6cdd77ec8e17ef71bf74a85792ed9ac29638d43"
Jul 17 18:21:28.408045 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:28.407809    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j_odh-ai-gateway-infra(f1c277a4-b8bb-4689-bd0d-c5edec30e1f0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0"
Jul 17 18:21:31.292339 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:31.292309    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cbb795d5f-vsc6t_e2f543b5-1949-4748-b631-bf9a52ab9052/copy-manifests/0.log"
Jul 17 18:21:31.298838 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:31.298814    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cbb795d5f-vsc6t_e2f543b5-1949-4748-b631-bf9a52ab9052/manager/0.log"
Jul 17 18:21:31.750109 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:31.750077    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738505-qqc2p_9557872a-ef96-4fa6-991f-eb43450860ce/cleanup/0.log"
Jul 17 18:21:31.862834 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:31.862807    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738520-v9r5f_7bd266de-a508-49f5-bb09-eb3508990517/cleanup/0.log"
Jul 17 18:21:31.972798 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:31.972768    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738535-lkngx_03ddc285-dad8-482c-986b-0553b9b70828/cleanup/0.log"
Jul 17 18:21:32.085673 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:32.085645    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-59516a0f-97vft_8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc/revoke-keys/1.log"
Jul 17 18:21:32.198185 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:32.198146    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-a27826-2md7x_60117f4f-e445-4065-9c9a-ba5a9b497653/revoke-keys/0.log"
Jul 17 18:21:32.309986 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:32.309955    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-5ebcf6-j2l5b_09d419b7-2667-477c-85c0-de7f88bb23ee/revoke-keys/0.log"
Jul 17 18:21:32.426552 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:32.426466    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j_f1c277a4-b8bb-4689-bd0d-c5edec30e1f0/revoke-keys/1.log"
Jul 17 18:21:32.534267 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:32.534239    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-54bdd1-5tw4d_96f6ed61-e321-4a30-ae79-4e9a9248e5cf/revoke-keys/0.log"
Jul 17 18:21:32.645559 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:32.645520    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-c50cdb-6snl7_dd8a0993-6dd7-445f-8bc1-3b914db314b3/revoke-keys/0.log"
Jul 17 18:21:32.757664 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:32.757587    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf_95a41ada-2fad-41c3-a449-99c7807099cb/revoke-keys/1.log"
Jul 17 18:21:32.866588 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:32.866562    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-a60650e8-8rw4p_710c0b14-159f-49ba-bca9-a86f3874c70a/revoke-keys/0.log"
Jul 17 18:21:32.973018 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:32.972987    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-f47fb1c3-ggwjj_9aac9f26-97d1-434d-867b-09e12388fd33/revoke-keys/0.log"
Jul 17 18:21:33.084044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:33.084015    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-4699ff-mnf2v_8b294fe6-3766-4604-9d48-d25322f79c70/revoke-keys/0.log"
Jul 17 18:21:33.193419 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:33.193385    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-22fb10-5lv9q_49cdf2b5-e249-4675-b8ad-8933b6d65d41/revoke-keys/0.log"
Jul 17 18:21:34.097901 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.097865    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k_ec935f99-40e0-4d29-84ce-ec44cde8f7e3/pull/0.log"
Jul 17 18:21:34.104668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.104641    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k_ec935f99-40e0-4d29-84ce-ec44cde8f7e3/extract/0.log"
Jul 17 18:21:34.110850 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.110825    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k_ec935f99-40e0-4d29-84ce-ec44cde8f7e3/util/0.log"
Jul 17 18:21:34.220647 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.220620    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h_47670189-5913-42dd-a2b9-aacef3d884af/pull/0.log"
Jul 17 18:21:34.227325 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.227294    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h_47670189-5913-42dd-a2b9-aacef3d884af/extract/0.log"
Jul 17 18:21:34.233652 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.233624    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h_47670189-5913-42dd-a2b9-aacef3d884af/util/0.log"
Jul 17 18:21:34.343090 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.343065    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8_941a052d-1fda-4c9c-9a51-3d2a6d6c5004/util/0.log"
Jul 17 18:21:34.350630 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.350551    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8_941a052d-1fda-4c9c-9a51-3d2a6d6c5004/pull/0.log"
Jul 17 18:21:34.357687 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.357664    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8_941a052d-1fda-4c9c-9a51-3d2a6d6c5004/extract/0.log"
Jul 17 18:21:34.464074 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.464041    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq_38e054be-1601-4e0e-ad96-a6748081b88d/util/0.log"
Jul 17 18:21:34.470853 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.470828    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq_38e054be-1601-4e0e-ad96-a6748081b88d/pull/0.log"
Jul 17 18:21:34.477255 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.477229    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq_38e054be-1601-4e0e-ad96-a6748081b88d/extract/0.log"
Jul 17 18:21:34.515548 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:34.515517    2572 scope.go:117] "RemoveContainer" containerID="4127ca7826c8562c69c56dbd430fe5683e1ab4f05dd3b3bfb13da626de2809ac"
Jul 17 18:21:35.154155 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:35.154125    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-t7j95_60f4ff0c-da55-44d0-9a49-b74260bb4a6c/manager/0.log"
Jul 17 18:21:35.450559 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:35.450469    2572 generic.go:356] "Generic (PLEG): container finished" podID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerID="8c9de8371a71e664695e3e610fa96c4d3671ed7dd8e5c45ee8fdb7de4577780e" exitCode=6
Jul 17 18:21:35.450559 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:35.450548    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" event={"ID":"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc","Type":"ContainerDied","Data":"8c9de8371a71e664695e3e610fa96c4d3671ed7dd8e5c45ee8fdb7de4577780e"}
Jul 17 18:21:35.450764 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:35.450612    2572 scope.go:117] "RemoveContainer" containerID="4127ca7826c8562c69c56dbd430fe5683e1ab4f05dd3b3bfb13da626de2809ac"
Jul 17 18:21:35.450889 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:35.450875    2572 scope.go:117] "RemoveContainer" containerID="8c9de8371a71e664695e3e610fa96c4d3671ed7dd8e5c45ee8fdb7de4577780e"
Jul 17 18:21:35.451098 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:35.451079    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-59516a0f-97vft_odh-ai-gateway-infra(8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc"
Jul 17 18:21:35.841031 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:35.840985    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xh54n_3cc37fd2-4752-410e-b411-9ff7aa84a0f0/discovery/0.log"
Jul 17 18:21:36.288845 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:36.288759    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jdrnd_a3e83a56-c4d7-4a82-a983-48c17c880fb0/payload-pre-processing/0.log"
Jul 17 18:21:36.409528 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:36.409475    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nn7kv_e2e1d612-7d04-4e29-9813-fd61d6e0a63a/payload-processing/0.log"
Jul 17 18:21:36.476692 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:36.476646    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"]
Jul 17 18:21:36.625747 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:36.625720    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"
Jul 17 18:21:36.665908 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:36.665872    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc-maas-api-service-ca\") pod \"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc\" (UID: \"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc\") "
Jul 17 18:21:36.666273 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:36.666245    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" (UID: "8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:21:36.767464 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:36.767417    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:21:36.887480 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:36.887401    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-dw47l_aa88a0b9-d2ad-40ae-8dba-1dec3938d94c/main/0.log"
Jul 17 18:21:37.455668 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:37.455634    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-fdc2j_70cf4691-f39b-445d-ab1a-0e04dcce50d5/main/0.log"
Jul 17 18:21:37.465301 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:37.465275    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"
Jul 17 18:21:37.465301 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:37.465291    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft" event={"ID":"8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc","Type":"ContainerDied","Data":"bc6766c94650531c7ad676f537a53b765d945b6734761463d39905f98d96e0a0"}
Jul 17 18:21:37.465567 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:37.465347    2572 scope.go:117] "RemoveContainer" containerID="8c9de8371a71e664695e3e610fa96c4d3671ed7dd8e5c45ee8fdb7de4577780e"
Jul 17 18:21:37.489269 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:37.489229    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"]
Jul 17 18:21:37.496287 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:37.496254    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-59516a0f-97vft"]
Jul 17 18:21:37.509613 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:37.509581    2572 scope.go:117] "RemoveContainer" containerID="90470de2f5f5b1fdb8fe0d0fbbde1408b10b41d54da90a8e4bbd981d716635f7"
Jul 17 18:21:38.473953 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:38.473856    2572 generic.go:356] "Generic (PLEG): container finished" podID="95a41ada-2fad-41c3-a449-99c7807099cb" containerID="8042f36e3f74e68e0b63022c898ef3a711ea55e0cb001c97b71c8ca6aa2dafb8" exitCode=6
Jul 17 18:21:38.473953 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:38.473929    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf" event={"ID":"95a41ada-2fad-41c3-a449-99c7807099cb","Type":"ContainerDied","Data":"8042f36e3f74e68e0b63022c898ef3a711ea55e0cb001c97b71c8ca6aa2dafb8"}
Jul 17 18:21:38.474439 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:38.473994    2572 scope.go:117] "RemoveContainer" containerID="90470de2f5f5b1fdb8fe0d0fbbde1408b10b41d54da90a8e4bbd981d716635f7"
Jul 17 18:21:38.474439 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:38.474263    2572 scope.go:117] "RemoveContainer" containerID="8042f36e3f74e68e0b63022c898ef3a711ea55e0cb001c97b71c8ca6aa2dafb8"
Jul 17 18:21:38.474577 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:38.474551    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf_odh-ai-gateway-infra(95a41ada-2fad-41c3-a449-99c7807099cb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf" podUID="95a41ada-2fad-41c3-a449-99c7807099cb"
Jul 17 18:21:38.516266 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:38.516227    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" path="/var/lib/kubelet/pods/8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc/volumes"
Jul 17 18:21:39.503121 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:39.503085    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"]
Jul 17 18:21:39.648974 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:39.648949    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"
Jul 17 18:21:39.692609 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:39.692576    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a41ada-2fad-41c3-a449-99c7807099cb-maas-api-service-ca\") pod \"95a41ada-2fad-41c3-a449-99c7807099cb\" (UID: \"95a41ada-2fad-41c3-a449-99c7807099cb\") "
Jul 17 18:21:39.692972 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:39.692945    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95a41ada-2fad-41c3-a449-99c7807099cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95a41ada-2fad-41c3-a449-99c7807099cb" (UID: "95a41ada-2fad-41c3-a449-99c7807099cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:21:39.793862 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:39.793769    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a41ada-2fad-41c3-a449-99c7807099cb-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:21:40.494618 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.494580    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"
Jul 17 18:21:40.494618 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.494608    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf" event={"ID":"95a41ada-2fad-41c3-a449-99c7807099cb","Type":"ContainerDied","Data":"0f890a00c0859c76c567930509f85a884f6a270bf87eaf7078a82502844babfb"}
Jul 17 18:21:40.494862 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.494670    2572 scope.go:117] "RemoveContainer" containerID="8042f36e3f74e68e0b63022c898ef3a711ea55e0cb001c97b71c8ca6aa2dafb8"
Jul 17 18:21:40.517841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.517810    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"]
Jul 17 18:21:40.521211 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.521178    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-36649ebd-64rzf"]
Jul 17 18:21:40.766848 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.766753    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"]
Jul 17 18:21:40.767121 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767108    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerName="revoke-keys"
Jul 17 18:21:40.767165 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767122    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerName="revoke-keys"
Jul 17 18:21:40.767165 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767133    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerName="revoke-keys"
Jul 17 18:21:40.767165 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767139    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerName="revoke-keys"
Jul 17 18:21:40.767165 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767146    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerName="revoke-keys"
Jul 17 18:21:40.767165 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767151    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerName="revoke-keys"
Jul 17 18:21:40.767324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767171    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" containerName="revoke-keys"
Jul 17 18:21:40.767324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767178    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" containerName="revoke-keys"
Jul 17 18:21:40.767324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767190    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" containerName="revoke-keys"
Jul 17 18:21:40.767324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767198    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" containerName="revoke-keys"
Jul 17 18:21:40.767324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767271    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerName="revoke-keys"
Jul 17 18:21:40.767324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767279    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" containerName="revoke-keys"
Jul 17 18:21:40.767324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767286    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerName="revoke-keys"
Jul 17 18:21:40.767324 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.767292    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" containerName="revoke-keys"
Jul 17 18:21:40.771550 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.771526    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"
Jul 17 18:21:40.777702 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.777670    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"]
Jul 17 18:21:40.903850 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:40.903810    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de17e71e-3257-46f2-8f6b-bc2815581c11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-jl472\" (UID: \"de17e71e-3257-46f2-8f6b-bc2815581c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"
Jul 17 18:21:41.005196 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:41.005157    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de17e71e-3257-46f2-8f6b-bc2815581c11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-jl472\" (UID: \"de17e71e-3257-46f2-8f6b-bc2815581c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"
Jul 17 18:21:41.005875 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:41.005853    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de17e71e-3257-46f2-8f6b-bc2815581c11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ce07281e-jl472\" (UID: \"de17e71e-3257-46f2-8f6b-bc2815581c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"
Jul 17 18:21:41.084732 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:41.084695    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"
Jul 17 18:21:41.230751 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:41.230718    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"]
Jul 17 18:21:41.232394 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:21:41.232365    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde17e71e_3257_46f2_8f6b_bc2815581c11.slice/crio-84d965cff74281389ca5392f22874df719f53e23c2bdf8c70276ce8bbe1a268a WatchSource:0}: Error finding container 84d965cff74281389ca5392f22874df719f53e23c2bdf8c70276ce8bbe1a268a: Status 404 returned error can't find the container with id 84d965cff74281389ca5392f22874df719f53e23c2bdf8c70276ce8bbe1a268a
Jul 17 18:21:41.502336 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:41.502300    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472" event={"ID":"de17e71e-3257-46f2-8f6b-bc2815581c11","Type":"ContainerStarted","Data":"84d965cff74281389ca5392f22874df719f53e23c2bdf8c70276ce8bbe1a268a"}
Jul 17 18:21:42.511222 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:42.511186    2572 generic.go:356] "Generic (PLEG): container finished" podID="de17e71e-3257-46f2-8f6b-bc2815581c11" containerID="b28c10a34d344952249bccb8a26f138a9082713a100234718152e01c6943a6d1" exitCode=6
Jul 17 18:21:42.511688 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:42.511587    2572 scope.go:117] "RemoveContainer" containerID="b28c10a34d344952249bccb8a26f138a9082713a100234718152e01c6943a6d1"
Jul 17 18:21:42.515989 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:42.515962    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" path="/var/lib/kubelet/pods/95a41ada-2fad-41c3-a449-99c7807099cb/volumes"
Jul 17 18:21:42.516345 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:42.516323    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472" event={"ID":"de17e71e-3257-46f2-8f6b-bc2815581c11","Type":"ContainerDied","Data":"b28c10a34d344952249bccb8a26f138a9082713a100234718152e01c6943a6d1"}
Jul 17 18:21:43.509908 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:43.509877    2572 scope.go:117] "RemoveContainer" containerID="f1b4db3871182fed1a845651f6cdd77ec8e17ef71bf74a85792ed9ac29638d43"
Jul 17 18:21:43.518913 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:43.518881    2572 generic.go:356] "Generic (PLEG): container finished" podID="de17e71e-3257-46f2-8f6b-bc2815581c11" containerID="980f41d60fbd8cd143995666ceef230b5122ccac8c9de81398a05cb944505c96" exitCode=6
Jul 17 18:21:43.519354 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:43.518962    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472" event={"ID":"de17e71e-3257-46f2-8f6b-bc2815581c11","Type":"ContainerDied","Data":"980f41d60fbd8cd143995666ceef230b5122ccac8c9de81398a05cb944505c96"}
Jul 17 18:21:43.519354 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:43.519019    2572 scope.go:117] "RemoveContainer" containerID="b28c10a34d344952249bccb8a26f138a9082713a100234718152e01c6943a6d1"
Jul 17 18:21:43.519354 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:43.519236    2572 scope.go:117] "RemoveContainer" containerID="980f41d60fbd8cd143995666ceef230b5122ccac8c9de81398a05cb944505c96"
Jul 17 18:21:43.519574 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:43.519522    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-jl472_odh-ai-gateway-infra(de17e71e-3257-46f2-8f6b-bc2815581c11)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472" podUID="de17e71e-3257-46f2-8f6b-bc2815581c11"
Jul 17 18:21:44.527315 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:44.527225    2572 scope.go:117] "RemoveContainer" containerID="980f41d60fbd8cd143995666ceef230b5122ccac8c9de81398a05cb944505c96"
Jul 17 18:21:44.527812 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:44.527454    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-jl472_odh-ai-gateway-infra(de17e71e-3257-46f2-8f6b-bc2815581c11)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472" podUID="de17e71e-3257-46f2-8f6b-bc2815581c11"
Jul 17 18:21:44.529091 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:44.529063    2572 generic.go:356] "Generic (PLEG): container finished" podID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" containerID="0ef05aaeb2503443f230742b74b6a71749cc72e6fa22332a150bf59e059ca24c" exitCode=6
Jul 17 18:21:44.529238 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:44.529137    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j" event={"ID":"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0","Type":"ContainerDied","Data":"0ef05aaeb2503443f230742b74b6a71749cc72e6fa22332a150bf59e059ca24c"}
Jul 17 18:21:44.529238 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:44.529210    2572 scope.go:117] "RemoveContainer" containerID="f1b4db3871182fed1a845651f6cdd77ec8e17ef71bf74a85792ed9ac29638d43"
Jul 17 18:21:44.529592 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:44.529577    2572 scope.go:117] "RemoveContainer" containerID="0ef05aaeb2503443f230742b74b6a71749cc72e6fa22332a150bf59e059ca24c"
Jul 17 18:21:44.529803 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:44.529785    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j_odh-ai-gateway-infra(f1c277a4-b8bb-4689-bd0d-c5edec30e1f0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0"
Jul 17 18:21:45.573149 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:45.573102    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"]
Jul 17 18:21:45.707473 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:45.707442    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"
Jul 17 18:21:45.851082 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:45.850980    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c277a4-b8bb-4689-bd0d-c5edec30e1f0-maas-api-service-ca\") pod \"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0\" (UID: \"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0\") "
Jul 17 18:21:45.851397 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:45.851373    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1c277a4-b8bb-4689-bd0d-c5edec30e1f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" (UID: "f1c277a4-b8bb-4689-bd0d-c5edec30e1f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:21:45.909508 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:45.909454    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p8nmp_7b5d7d3c-465d-40c3-8b32-d303103c7dd9/global-pull-secret-syncer/0.log"
Jul 17 18:21:45.952108 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:45.952070    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c277a4-b8bb-4689-bd0d-c5edec30e1f0-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:21:46.031709 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:46.031671    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vbj8d_ed5123f0-04f4-45e5-8098-6827fe05c426/konnectivity-agent/0.log"
Jul 17 18:21:46.075682 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:46.075650    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-30.ec2.internal_73ecd14a99b2d4ac78d23a5262bad321/haproxy/0.log"
Jul 17 18:21:46.544625 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:46.544591    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"
Jul 17 18:21:46.544823 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:46.544591    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j" event={"ID":"f1c277a4-b8bb-4689-bd0d-c5edec30e1f0","Type":"ContainerDied","Data":"e634ebc145dd0953120e2cc3cdc2ede9fca1b41eb41f08d4f72391d22a7dd8b7"}
Jul 17 18:21:46.544823 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:46.544727    2572 scope.go:117] "RemoveContainer" containerID="0ef05aaeb2503443f230742b74b6a71749cc72e6fa22332a150bf59e059ca24c"
Jul 17 18:21:46.562161 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:46.562128    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"]
Jul 17 18:21:46.564641 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:46.564619    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e49d4a9a-p4d5j"]
Jul 17 18:21:48.516207 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:48.516171    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" path="/var/lib/kubelet/pods/f1c277a4-b8bb-4689-bd0d-c5edec30e1f0/volumes"
Jul 17 18:21:49.355872 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:49.355836    2572 scope.go:117] "RemoveContainer" containerID="06095cbb59c5cfba1abaaf5ec6ffd3a3576311dc7c60f6fced17d5c0f733eeeb"
Jul 17 18:21:49.365013 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:49.364990    2572 scope.go:117] "RemoveContainer" containerID="1ea969402a2745139a6e91988ad11a55b7054ba354f7825b56b95cff07217014"
Jul 17 18:21:50.341758 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.341725    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k_ec935f99-40e0-4d29-84ce-ec44cde8f7e3/extract/0.log"
Jul 17 18:21:50.374803 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.374766    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k_ec935f99-40e0-4d29-84ce-ec44cde8f7e3/util/0.log"
Jul 17 18:21:50.396953 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.396925    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gcz7k_ec935f99-40e0-4d29-84ce-ec44cde8f7e3/pull/0.log"
Jul 17 18:21:50.428252 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.428220    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h_47670189-5913-42dd-a2b9-aacef3d884af/extract/0.log"
Jul 17 18:21:50.452155 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.452127    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h_47670189-5913-42dd-a2b9-aacef3d884af/util/0.log"
Jul 17 18:21:50.476779 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.476754    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5t9h_47670189-5913-42dd-a2b9-aacef3d884af/pull/0.log"
Jul 17 18:21:50.505794 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.505764    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8_941a052d-1fda-4c9c-9a51-3d2a6d6c5004/extract/0.log"
Jul 17 18:21:50.534929 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.534897    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8_941a052d-1fda-4c9c-9a51-3d2a6d6c5004/util/0.log"
Jul 17 18:21:50.558895 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.558864    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73svdh8_941a052d-1fda-4c9c-9a51-3d2a6d6c5004/pull/0.log"
Jul 17 18:21:50.608080 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.607992    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq_38e054be-1601-4e0e-ad96-a6748081b88d/extract/0.log"
Jul 17 18:21:50.635541 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.635486    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq_38e054be-1601-4e0e-ad96-a6748081b88d/util/0.log"
Jul 17 18:21:50.659627 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:50.659597    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cztlq_38e054be-1601-4e0e-ad96-a6748081b88d/pull/0.log"
Jul 17 18:21:51.428520 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:51.428460    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-t7j95_60f4ff0c-da55-44d0-9a49-b74260bb4a6c/manager/0.log"
Jul 17 18:21:53.184466 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:53.184422    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-p8rl8_642842c7-dd0b-4a6a-a358-eeea433a7d25/cluster-monitoring-operator/0.log"
Jul 17 18:21:53.454191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:53.454101    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g24rs_450ae286-b9a8-4e77-a616-46040d067be1/node-exporter/0.log"
Jul 17 18:21:53.478428 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:53.478401    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g24rs_450ae286-b9a8-4e77-a616-46040d067be1/kube-rbac-proxy/0.log"
Jul 17 18:21:53.502982 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:53.502956    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g24rs_450ae286-b9a8-4e77-a616-46040d067be1/init-textfile/0.log"
Jul 17 18:21:55.784789 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:55.784757    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-98j2x_638d220b-0afb-46b6-8826-4110aeb4940b/networking-console-plugin/0.log"
Jul 17 18:21:56.250607 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.250564    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"]
Jul 17 18:21:56.250981 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.250965    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" containerName="revoke-keys"
Jul 17 18:21:56.251044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.250983    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" containerName="revoke-keys"
Jul 17 18:21:56.251044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.251005    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" containerName="revoke-keys"
Jul 17 18:21:56.251044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.251010    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" containerName="revoke-keys"
Jul 17 18:21:56.251044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.251022    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" containerName="revoke-keys"
Jul 17 18:21:56.251044 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.251028    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" containerName="revoke-keys"
Jul 17 18:21:56.251191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.251087    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" containerName="revoke-keys"
Jul 17 18:21:56.251191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.251095    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8ea6c7-97aa-44d4-8cae-fa7a6e31cafc" containerName="revoke-keys"
Jul 17 18:21:56.251191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.251100    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a41ada-2fad-41c3-a449-99c7807099cb" containerName="revoke-keys"
Jul 17 18:21:56.251191 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.251106    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1c277a4-b8bb-4689-bd0d-c5edec30e1f0" containerName="revoke-keys"
Jul 17 18:21:56.255983 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.255962    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.258653 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.258619    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5g29d\"/\"openshift-service-ca.crt\""
Jul 17 18:21:56.258831 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.258621    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5g29d\"/\"default-dockercfg-rgpp8\""
Jul 17 18:21:56.259828 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.259809    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5g29d\"/\"kube-root-ca.crt\""
Jul 17 18:21:56.265378 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.265356    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"]
Jul 17 18:21:56.349688 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.349644    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-proc\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.349885 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.349710    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-sys\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.349885 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.349742    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-podres\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.349885 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.349785    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-lib-modules\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.350003 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.349899    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt9cr\" (UniqueName: \"kubernetes.io/projected/1342865a-4215-4a88-a08c-c08e215c594d-kube-api-access-qt9cr\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.450560 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.450482    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt9cr\" (UniqueName: \"kubernetes.io/projected/1342865a-4215-4a88-a08c-c08e215c594d-kube-api-access-qt9cr\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.450716 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.450585    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-proc\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.450716 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.450625    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-sys\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.450716 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.450642    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-podres\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.450716 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.450664    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-lib-modules\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.450910 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.450720    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-proc\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.450910 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.450726    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-sys\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.450910 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.450796    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-lib-modules\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.450910 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.450817    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1342865a-4215-4a88-a08c-c08e215c594d-podres\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.460176 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.460137    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt9cr\" (UniqueName: \"kubernetes.io/projected/1342865a-4215-4a88-a08c-c08e215c594d-kube-api-access-qt9cr\") pod \"perf-node-gather-daemonset-s57kf\" (UID: \"1342865a-4215-4a88-a08c-c08e215c594d\") " pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.490567 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.490532    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/1.log"
Jul 17 18:21:56.496338 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.496309    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p75lc_26681e78-0726-442e-8390-d6c9917d77da/console-operator/2.log"
Jul 17 18:21:56.567654 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.567619    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:56.711725 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:56.711698    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"]
Jul 17 18:21:56.713601 ip-10-0-135-30 kubenswrapper[2572]: W0717 18:21:56.713571    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1342865a_4215_4a88_a08c_c08e215c594d.slice/crio-c8979a6b52da1ce11ee26c6dfb40961e8a630d80bc67661afd06e4a16ecb9e7d WatchSource:0}: Error finding container c8979a6b52da1ce11ee26c6dfb40961e8a630d80bc67661afd06e4a16ecb9e7d: Status 404 returned error can't find the container with id c8979a6b52da1ce11ee26c6dfb40961e8a630d80bc67661afd06e4a16ecb9e7d
Jul 17 18:21:57.054593 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:57.054555    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-jmwvg_3c455fa4-d71d-4748-abe2-6c903f6ae0f3/download-server/0.log"
Jul 17 18:21:57.510077 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:57.510047    2572 scope.go:117] "RemoveContainer" containerID="980f41d60fbd8cd143995666ceef230b5122ccac8c9de81398a05cb944505c96"
Jul 17 18:21:57.617348 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:57.617313    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf" event={"ID":"1342865a-4215-4a88-a08c-c08e215c594d","Type":"ContainerStarted","Data":"bb3d24469a0552c52dbd4fd2f23232d51e360575b8a25f0b993f669a7c8431fd"}
Jul 17 18:21:57.617557 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:57.617366    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf" event={"ID":"1342865a-4215-4a88-a08c-c08e215c594d","Type":"ContainerStarted","Data":"c8979a6b52da1ce11ee26c6dfb40961e8a630d80bc67661afd06e4a16ecb9e7d"}
Jul 17 18:21:57.617557 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:57.617505    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:21:57.638243 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:57.638175    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf" podStartSLOduration=1.638155409 podStartE2EDuration="1.638155409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:21:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:21:57.635518128 +0000 UTC m=+3489.640873977" watchObservedRunningTime="2026-07-17 18:21:57.638155409 +0000 UTC m=+3489.643511264"
Jul 17 18:21:57.657728 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:57.657696    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-9fxx4_f5dd4a69-b165-44db-91e8-3317b4f37dd0/volume-data-source-validator/0.log"
Jul 17 18:21:58.625617 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:58.625581    2572 generic.go:356] "Generic (PLEG): container finished" podID="de17e71e-3257-46f2-8f6b-bc2815581c11" containerID="5cac1687ec13dd6af1fad570b09382c176a3efe8f203f1636d039cbf564c59b5" exitCode=6
Jul 17 18:21:58.626121 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:58.625653    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472" event={"ID":"de17e71e-3257-46f2-8f6b-bc2815581c11","Type":"ContainerDied","Data":"5cac1687ec13dd6af1fad570b09382c176a3efe8f203f1636d039cbf564c59b5"}
Jul 17 18:21:58.626121 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:58.625725    2572 scope.go:117] "RemoveContainer" containerID="980f41d60fbd8cd143995666ceef230b5122ccac8c9de81398a05cb944505c96"
Jul 17 18:21:58.626121 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:58.626047    2572 scope.go:117] "RemoveContainer" containerID="5cac1687ec13dd6af1fad570b09382c176a3efe8f203f1636d039cbf564c59b5"
Jul 17 18:21:58.626323 ip-10-0-135-30 kubenswrapper[2572]: E0717 18:21:58.626265    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ce07281e-jl472_odh-ai-gateway-infra(de17e71e-3257-46f2-8f6b-bc2815581c11)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472" podUID="de17e71e-3257-46f2-8f6b-bc2815581c11"
Jul 17 18:21:58.714841 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:58.714813    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-74tnw_de66d357-8e8a-4315-9db4-9adf5d86d420/dns/0.log"
Jul 17 18:21:58.740900 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:58.740865    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-74tnw_de66d357-8e8a-4315-9db4-9adf5d86d420/kube-rbac-proxy/0.log"
Jul 17 18:21:58.889043 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:58.888954    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l2pbp_12657b07-032c-4033-88ac-46e27583c859/dns-node-resolver/0.log"
Jul 17 18:21:59.475686 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:59.475646    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l9hsk_1fb7e912-6649-48cc-88c7-989f172f937c/node-ca/0.log"
Jul 17 18:21:59.651424 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:59.651383    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"]
Jul 17 18:21:59.789100 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:59.789071    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"
Jul 17 18:21:59.880805 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:59.880767    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de17e71e-3257-46f2-8f6b-bc2815581c11-maas-api-service-ca\") pod \"de17e71e-3257-46f2-8f6b-bc2815581c11\" (UID: \"de17e71e-3257-46f2-8f6b-bc2815581c11\") "
Jul 17 18:21:59.881158 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:59.881134    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de17e71e-3257-46f2-8f6b-bc2815581c11-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de17e71e-3257-46f2-8f6b-bc2815581c11" (UID: "de17e71e-3257-46f2-8f6b-bc2815581c11"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:21:59.981419 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:21:59.981380    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de17e71e-3257-46f2-8f6b-bc2815581c11-maas-api-service-ca\") on node \"ip-10-0-135-30.ec2.internal\" DevicePath \"\""
Jul 17 18:22:00.574146 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:00.574111    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xh54n_3cc37fd2-4752-410e-b411-9ff7aa84a0f0/discovery/0.log"
Jul 17 18:22:00.640374 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:00.640343    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"
Jul 17 18:22:00.640609 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:00.640385    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472" event={"ID":"de17e71e-3257-46f2-8f6b-bc2815581c11","Type":"ContainerDied","Data":"84d965cff74281389ca5392f22874df719f53e23c2bdf8c70276ce8bbe1a268a"}
Jul 17 18:22:00.640609 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:00.640451    2572 scope.go:117] "RemoveContainer" containerID="5cac1687ec13dd6af1fad570b09382c176a3efe8f203f1636d039cbf564c59b5"
Jul 17 18:22:00.657719 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:00.657681    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"]
Jul 17 18:22:00.663767 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:00.663728    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ce07281e-jl472"]
Jul 17 18:22:00.722395 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:00.722355    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jdrnd_a3e83a56-c4d7-4a82-a983-48c17c880fb0/payload-pre-processing/0.log"
Jul 17 18:22:00.752415 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:00.752383    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nn7kv_e2e1d612-7d04-4e29-9813-fd61d6e0a63a/payload-processing/0.log"
Jul 17 18:22:01.511604 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:01.511575    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-djgz7_f15bcab1-4a49-45b7-94a7-f7175f8ce2c5/serve-healthcheck-canary/0.log"
Jul 17 18:22:02.040963 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:02.040925    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-kgqwx_bcf3189c-2328-4a13-85f6-0cb7423020cc/insights-operator/0.log"
Jul 17 18:22:02.041924 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:02.041909    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-kgqwx_bcf3189c-2328-4a13-85f6-0cb7423020cc/insights-operator/1.log"
Jul 17 18:22:02.084520 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:02.084468    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d9xdg_e98b193a-4711-4e94-b692-d0d36fe172d3/kube-rbac-proxy/0.log"
Jul 17 18:22:02.107111 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:02.107082    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d9xdg_e98b193a-4711-4e94-b692-d0d36fe172d3/exporter/0.log"
Jul 17 18:22:02.130056 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:02.130023    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d9xdg_e98b193a-4711-4e94-b692-d0d36fe172d3/extractor/0.log"
Jul 17 18:22:02.516105 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:02.516068    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de17e71e-3257-46f2-8f6b-bc2815581c11" path="/var/lib/kubelet/pods/de17e71e-3257-46f2-8f6b-bc2815581c11/volumes"
Jul 17 18:22:03.636484 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:03.636455    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5g29d/perf-node-gather-daemonset-s57kf"
Jul 17 18:22:05.022532 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:05.022477    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cbb795d5f-vsc6t_e2f543b5-1949-4748-b631-bf9a52ab9052/manager/0.log"
Jul 17 18:22:05.045110 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:05.045084    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cbb795d5f-vsc6t_e2f543b5-1949-4748-b631-bf9a52ab9052/copy-manifests/0.log"
Jul 17 18:22:06.334184 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:06.334149    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xh54n_3cc37fd2-4752-410e-b411-9ff7aa84a0f0/discovery/0.log"
Jul 17 18:22:06.443733 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:06.443695    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jdrnd_a3e83a56-c4d7-4a82-a983-48c17c880fb0/payload-pre-processing/0.log"
Jul 17 18:22:06.452854 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:06.452828    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d77fc5c7f-9srws_817a2fe7-7397-4e72-b89a-3cd732c2dc43/manager/0.log"
Jul 17 18:22:06.463576 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:06.463545    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nn7kv_e2e1d612-7d04-4e29-9813-fd61d6e0a63a/payload-processing/0.log"
Jul 17 18:22:07.745823 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:07.745788    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xh54n_3cc37fd2-4752-410e-b411-9ff7aa84a0f0/discovery/0.log"
Jul 17 18:22:07.865242 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:07.865208    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jdrnd_a3e83a56-c4d7-4a82-a983-48c17c880fb0/payload-pre-processing/0.log"
Jul 17 18:22:07.886125 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:07.886037    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nn7kv_e2e1d612-7d04-4e29-9813-fd61d6e0a63a/payload-processing/0.log"
Jul 17 18:22:13.141012 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:13.140975    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2xkgl_bd33444c-515f-4f4d-a793-933561aee26d/kube-multus/0.log"
Jul 17 18:22:13.415524 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:13.415420    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qgc79_e32df813-b6ae-45ff-9b71-82510269769b/kube-multus-additional-cni-plugins/0.log"
Jul 17 18:22:13.439325 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:13.439298    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qgc79_e32df813-b6ae-45ff-9b71-82510269769b/egress-router-binary-copy/0.log"
Jul 17 18:22:13.463145 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:13.463122    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qgc79_e32df813-b6ae-45ff-9b71-82510269769b/cni-plugins/0.log"
Jul 17 18:22:13.485959 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:13.485935    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qgc79_e32df813-b6ae-45ff-9b71-82510269769b/bond-cni-plugin/0.log"
Jul 17 18:22:13.511396 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:13.511372    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qgc79_e32df813-b6ae-45ff-9b71-82510269769b/routeoverride-cni/0.log"
Jul 17 18:22:13.536413 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:13.536374    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qgc79_e32df813-b6ae-45ff-9b71-82510269769b/whereabouts-cni-bincopy/0.log"
Jul 17 18:22:13.560389 ip-10-0-135-30 kubenswrapper[2572]: I0717 18:22:13.560357    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qgc79_e32df813-b6ae-45ff-9b71-82510269769b/whereabouts-cni/0.log"