Jul 17 10:55:16.658152 ip-10-0-134-138 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 17 10:55:16.658167 ip-10-0-134-138 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 17 10:55:16.658177 ip-10-0-134-138 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 17 10:55:16.658490 ip-10-0-134-138 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 17 10:55:26.766666 ip-10-0-134-138 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 17 10:55:26.766685 ip-10-0-134-138 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e4759f02c61c4fc28e8260b69bb43324 --
Jul 17 10:56:56.998998 ip-10-0-134-138 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.470997    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479149    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479161    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479165    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479168    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479171    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479174    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479177    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 10:56:57.479863 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479180    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479183    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479185    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479188    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479191    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479194    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479197    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479199    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479202    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479204    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479206    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479209    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479212    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479215    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479218    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479220    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479224    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479226    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479229    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479232    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 10:56:57.494089 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479235    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479238    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479241    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479243    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479246    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479248    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479251    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479254    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479256    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479259    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479262    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479266    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479270    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479273    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479276    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479279    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479282    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479285    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479288    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 10:56:57.506354 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479291    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479294    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479297    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479300    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479303    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479305    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479308    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479311    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479314    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479317    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479319    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479322    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479325    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479328    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479331    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479334    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479336    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479340    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479344    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 10:56:57.509630 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479347    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479351    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479356    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479360    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479362    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479365    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479368    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479371    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479374    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479376    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479379    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479382    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479385    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479387    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479390    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479392    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479395    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479397    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479399    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479402    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 10:56:57.523201 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479405    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479407    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479410    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479412    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479415    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479417    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479420    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479422    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479425    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479429    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479432    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479435    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479438    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.479441    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481727    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481735    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481738    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481742    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481745    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481749    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 10:56:57.639326 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481752    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 10:56:57.555885 ip-10-0-134-138 systemd[1]: Started Kubernetes Kubelet.
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481755    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481758    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481760    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481763    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481765    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481768    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481770    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481773    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481775    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481778    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481781    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481783    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481786    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481788    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481791    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481793    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481796    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481799    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481801    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 10:56:57.787512 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481804    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481806    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481809    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481811    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481814    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481817    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481823    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481827    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481830    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481833    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481836    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481839    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481842    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481845    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481848    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481850    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481853    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481856    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481858    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481861    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 10:56:57.794476 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481863    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481866    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481868    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481871    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481873    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481875    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481878    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481880    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481883    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481886    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481888    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481890    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481893    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481895    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481898    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481902    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481906    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481910    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481914    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 10:56:57.795237 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481917    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481920    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481924    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481928    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481930    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481933    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481936    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481939    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481941    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481944    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481947    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481949    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481952    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481954    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481956    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481959    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481962    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481964    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481966    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 10:56:57.795825 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481969    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481971    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481974    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481977    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481979    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481982    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481985    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481988    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481990    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481993    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481996    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.481998    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482003    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482005    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482008    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482397    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482404    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482407    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482410    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482413    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 10:56:57.796422 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482415    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482418    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482420    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482423    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482426    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482428    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482431    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482433    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482438    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482442    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482444    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482447    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482450    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482453    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482456    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482458    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482461    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482463    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482466    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 10:56:57.797022 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482468    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482471    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482473    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482476    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482479    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482484    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482487    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482490    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482494    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482497    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482500    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482503    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482505    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482508    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482511    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482513    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482516    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482518    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482521    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482523    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 10:56:57.797631 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482526    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482529    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482532    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482534    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482537    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482539    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482542    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482544    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482547    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482549    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482553    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482556    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482558    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482560    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482563    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482565    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482571    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482575    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482579    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482582    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 10:56:57.798399 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482585    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482587    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482590    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482593    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482596    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482598    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482601    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482604    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482607    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482609    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482612    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482614    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482617    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482619    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482621    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482624    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482627    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482630    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482632    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482635    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 10:56:57.799053 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482637    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482639    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482642    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482645    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482647    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482650    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482653    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482655    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482659    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482663    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482666    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482668    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482671    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482673    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.482676    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482771    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482781    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482791    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482796    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482801    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482805    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 10:56:57.799686 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482809    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482813    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482816    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482819    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482823    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482826    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482829    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482832    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482835    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482838    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482841    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482844    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482848    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482851    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482854    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482857    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482860    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482864    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482867    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482871    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482876    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482879    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482882    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482884    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 10:56:57.800367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482888    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482891    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482895    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482898    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482901    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482904    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482907    2566 flags.go:64] FLAG: --enable-server="true"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482910    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482916    2566 flags.go:64] FLAG: --event-burst="100"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482919    2566 flags.go:64] FLAG: --event-qps="50"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482922    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482925    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482928    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482931    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482934    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482937    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482941    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482944    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482947    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482949    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482952    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482955    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482958    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482960    2566 flags.go:64] FLAG: --feature-gates=""
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482964    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 10:56:57.803551 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482967    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482970    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482975    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482979    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482982    2566 flags.go:64] FLAG: --help="false"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482985    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-134-138.ec2.internal"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482988    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482991    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482994    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.482998    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483002    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483005    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483008    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483010    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483013    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483017    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483020    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483023    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483025    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483028    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483031    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483034    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483037    2566 flags.go:64] FLAG: --lock-file=""
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483040    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 10:56:57.805028 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483043    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483045    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483050    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483053    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483056    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483059    2566 flags.go:64] FLAG: --logging-format="text"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483062    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483065    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483068    2566 flags.go:64] FLAG: --manifest-url=""
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483071    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483077    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483081    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483085    2566 flags.go:64] FLAG: --max-pods="110"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483088    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483091    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483094    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483097    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483100    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483102    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483106    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483113    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483116    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483119    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 10:56:57.807675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483122    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483125    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483130    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483133    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483136    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483139    2566 flags.go:64] FLAG: --port="10250"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483142    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483145    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-042d877a4287e6cc8"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483148    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483150    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483153    2566 flags.go:64] FLAG: --register-node="true"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483156    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483160    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483163    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483165    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483168    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483172    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483175    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483178    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483182    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483185    2566 flags.go:64] FLAG: --runonce="false"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483189    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483192    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483196    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483199    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 10:56:57.810209 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483201    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483205    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483208    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483211    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483213    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483216    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483219    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483221    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483224    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483228    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483231    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483236    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483238    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483242    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483246    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483248    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483251    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483254    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483257    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483260    2566 flags.go:64] FLAG: --v="2"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483263    2566 flags.go:64] FLAG: --version="false"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483267    2566 flags.go:64] FLAG: --vmodule=""
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483276    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.483279    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483389    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 10:56:57.813287 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483393    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483397    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483400    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483404    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483407    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483409    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483412    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483415    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483417    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483420    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483422    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483425    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483427    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483430    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483432    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483435    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483437    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483441    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483443    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 10:56:57.816381 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483447    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483451    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483454    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483457    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483460    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483462    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483465    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483468    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483470    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483473    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483475    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483478    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483480    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483483    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483487    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483490    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483495    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483497    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483500    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483502    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 10:56:57.817432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483505    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483507    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483510    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483512    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483515    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483517    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483520    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483523    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483525    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483528    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483530    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483533    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483536    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483539    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483541    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483544    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483546    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483549    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483551    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483554    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 10:56:57.818432 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483556    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483559    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483561    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483564    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483566    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483570    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483575    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483578    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483582    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483586    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483590    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483593    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483596    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483599    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483602    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483604    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483607    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483609    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483612    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 10:56:57.819330 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483615    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483618    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483620    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483622    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483625    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483628    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483630    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483633    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483636    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483638    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483641    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483643    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483646    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483648    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483651    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483654    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483656    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483659    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483661    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 10:56:57.820151 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.483665    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.484546    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.491763    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.491780    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.491794    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.491799    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491865    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491871    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491875    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491878    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491880    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491884    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491886    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491889    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 10:56:57.918520 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491891    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491894    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491897    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491899    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491902    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491904    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491907    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491910    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491912    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491915    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491918    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491922    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491929    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491933    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491937    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491940    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491943    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491946    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491949    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 10:56:57.919143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491952    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491955    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491963    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491966    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491969    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491972    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491974    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491977    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491980    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491982    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491985    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491987    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491990    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491992    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491995    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.491997    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492000    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492002    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492005    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492008    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 10:56:57.924906 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492011    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492013    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492015    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492018    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492021    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492025    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492028    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492030    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492033    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492035    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492038    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492041    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492043    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492046    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492048    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492057    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492061    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492064    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492067    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 10:56:57.925479 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492070    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492073    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492076    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492078    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492081    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492084    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492087    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492090    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492092    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492095    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492098    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492100    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492103    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492106    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492108    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492111    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492113    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492116    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492121    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492124    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 10:56:57.926016 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492126    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492129    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492131    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492134    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492136    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492139    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492142    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492144    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492147    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492156    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492159    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492162    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492164    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.492175    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492316    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 10:56:57.926585 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492321    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492324    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492327    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492330    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492333    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492336    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492339    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492341    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492344    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492346    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492349    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492351    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492354    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492356    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492359    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492363    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492367    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492369    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492372    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492375    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 10:56:57.927135 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492377    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492380    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492382    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492385    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492387    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492390    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492392    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492400    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492403    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492406    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492408    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492411    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492415    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492418    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492421    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492424    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492427    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492430    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492432    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 10:56:57.927700 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492435    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492437    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492440    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492442    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492445    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492447    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492449    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492452    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492456    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492461    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492464    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492466    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492469    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492471    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492474    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492477    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492479    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492482    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492484    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492487    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 10:56:57.928223 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492490    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492498    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492501    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492503    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492506    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492509    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492513    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492515    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492518    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492520    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492523    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492525    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492528    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492530    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492532    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492535    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492537    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492540    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492543    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492545    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 10:56:57.928827 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492550    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492553    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492556    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492559    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492561    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492564    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492568    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492571    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492574    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492577    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492579    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492582    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492585    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492588    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492596    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492598    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492601    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492603    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 10:56:57.929529 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:56:57.492606    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.492611    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.492743    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.495238    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.497289    2566 server.go:1019] "Starting client certificate rotation"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.497396    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.497441    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.500354    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.504818    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.514226    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.516735    2566 log.go:25] "Validated CRI v1 image API"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.517602    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 10:56:57.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.522598    2566 fs.go:135] Filesystem UUIDs: map[242f5e02-825f-45ad-839b-5ce5bd98c9e9:/dev/nvme0n1p3 650ed887-5853-44dc-ac1a-9027454bcdc6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 17 10:56:57.930618 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.522625    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 10:56:58.014405 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.529879    2566 manager.go:216] Machine: {Timestamp:2026-07-17 10:56:57.528412467 +0000 UTC m=+0.385496403 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099126 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec252eb30cd95e8ff778e8c4b4340074 SystemUUID:ec252eb3-0cd9-5e8f-f778-e8c4b4340074 BootID:e4759f02-c61c-4fc2-8e82-60b69bb43324 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:02:02:87:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:02:02:87:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:2c:e4:a6:92:b3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 10:56:58.014405 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.529976    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 10:56:58.014405 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.530057    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 10:56:58.014405 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.533550    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 10:56:58.014405 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.533580    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-138.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 10:56:58.014405 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.534486    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 10:56:58.014405 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.534498    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 10:56:58.014405 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.534506    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.534520    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.534795    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.536257    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.539762    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.539886    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.539911    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.539920    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.539934    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.539943    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.539951    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.540371    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.540394    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.542793    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.545480    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.545507    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546884    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546897    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546904    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546918    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546928    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546937    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546946    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546951    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546957    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546962    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546979    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.546985    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.547781    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.547789    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.550890    2566 server.go:1268] "Started kubelet"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.550942    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.555214    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.555292    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.555651    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-138.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.556166    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-138.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.556371    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.559824    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.562483    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 10:56:58.015607 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.563180    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.564331    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-138.ec2.internal.18c30e7f5ce2f4f9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-138.ec2.internal,UID:ip-10-0-134-138.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-138.ec2.internal,},FirstTimestamp:2026-07-17 10:56:57.550861561 +0000 UTC m=+0.407945496,LastTimestamp:2026-07-17 10:56:57.550861561 +0000 UTC m=+0.407945496,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-138.ec2.internal,}"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.569124    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.569185    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.569266    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.569819    2566 factory.go:55] Registering systemd factory
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.569832    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.569949    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.569950    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.569981    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.570037    2566 factory.go:153] Registering CRI-O factory
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.570051    2566 factory.go:223] Registration of the crio container factory successfully
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.570068    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.570076    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.570293    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.570315    2566 factory.go:103] Registering Raw factory
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.570330    2566 manager.go:1196] Started watching for new ooms in manager
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.571046    2566 manager.go:349] Starting recovery of all containers
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.571833    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.571910    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-138.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.579721    2566 manager.go:354] Recovery completed
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.584140    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.585900    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.585934    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.585944    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.586418    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.586429    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.586447    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.586419    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.588384    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.588411    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.588426    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.588841    2566 policy_none.go:49] "None policy: Start"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.588854    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 10:56:58.016742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.588864    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.589831    2566 policy_none.go:47] "Start"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.594032    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lfq46"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.596352    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-138.ec2.internal.18c30e7f5ef9eb5f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-138.ec2.internal,UID:ip-10-0-134-138.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-138.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-138.ec2.internal,},FirstTimestamp:2026-07-17 10:56:57.585920863 +0000 UTC m=+0.443004799,LastTimestamp:2026-07-17 10:56:57.585920863 +0000 UTC m=+0.443004799,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-138.ec2.internal,}"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.597839    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-138.ec2.internal.18c30e7f5efa3039  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-138.ec2.internal,UID:ip-10-0-134-138.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-138.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-138.ec2.internal,},FirstTimestamp:2026-07-17 10:56:57.585938489 +0000 UTC m=+0.443022425,LastTimestamp:2026-07-17 10:56:57.585938489 +0000 UTC m=+0.443022425,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-138.ec2.internal,}"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.602901    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lfq46"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.649822    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.650952    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.650985    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.651003    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.651033    2566 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.653222    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.656237    2566 manager.go:337] "Starting Device Plugin manager"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.656264    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.656272    2566 server.go:87] "Starting device plugin registration server"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.656531    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.656542    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.656658    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.656739    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.656749    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.657199    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.657261    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.751325    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"]
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.751421    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.753107    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 10:56:58.017952 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.753142    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.753155    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.757254    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.759966    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.759995    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.760011    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.760036    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.771525    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/105a462c32e44b70fe7bbcf79e028e08-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"105a462c32e44b70fe7bbcf79e028e08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.771553    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/105a462c32e44b70fe7bbcf79e028e08-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"105a462c32e44b70fe7bbcf79e028e08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.772635    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.772650    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-138.ec2.internal\": node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.787907    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.787972    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.788166    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.789202    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.789236    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.789255    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.789291    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.789310    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.789318    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.790650    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.790676    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.791408    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.791433    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.791444    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.801005    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.018766 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.872566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/105a462c32e44b70fe7bbcf79e028e08-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"105a462c32e44b70fe7bbcf79e028e08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.872605    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/105a462c32e44b70fe7bbcf79e028e08-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"105a462c32e44b70fe7bbcf79e028e08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.872626    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47af943b4b7e55afd17e31a4e269c392-config\") pod \"kube-apiserver-proxy-ip-10-0-134-138.ec2.internal\" (UID: \"47af943b4b7e55afd17e31a4e269c392\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.872661    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/105a462c32e44b70fe7bbcf79e028e08-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"105a462c32e44b70fe7bbcf79e028e08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.872670    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/105a462c32e44b70fe7bbcf79e028e08-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"105a462c32e44b70fe7bbcf79e028e08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.901938    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.941670    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-138.ec2.internal\" not found" node="ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:57.946006    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-138.ec2.internal\" not found" node="ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.955876    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.973266    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47af943b4b7e55afd17e31a4e269c392-config\") pod \"kube-apiserver-proxy-ip-10-0-134-138.ec2.internal\" (UID: \"47af943b4b7e55afd17e31a4e269c392\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:57.973311    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47af943b4b7e55afd17e31a4e269c392-config\") pod \"kube-apiserver-proxy-ip-10-0-134-138.ec2.internal\" (UID: \"47af943b4b7e55afd17e31a4e269c392\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.019582 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:58.002417    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.131358 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:58.103103    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.260090 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:58.203570    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.260090 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.249768    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.369240 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:58.304650    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.507937 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:58.405145    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.507937 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.434010    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 10:56:58.507937 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.496310    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 10:56:58.507937 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.496432    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 10:56:58.507937 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.496448    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 10:56:58.507937 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:58.505493    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 17 10:56:58.624194 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.593386    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 10:56:58.624194 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.607569    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 10:51:57 +0000 UTC" deadline="2028-01-04 18:29:48.326077635 +0000 UTC"
Jul 17 10:56:58.624194 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.607602    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12871h32m49.718479103s"
Jul 17 10:56:58.753584 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.669324    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.753584 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.681874    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 10:56:58.753584 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.683120    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 17 10:56:58.753584 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:58.697623    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 10:56:59.131466 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.107021    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 10:56:59.543348 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.541531    2566 apiserver.go:52] "Watching apiserver"
Jul 17 10:56:59.559985 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.554057    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 10:56:59.559985 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.555430    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-m86vn","openshift-network-operator/iptables-alerter-gb5wc","openshift-ovn-kubernetes/ovnkube-node-sl2kk","kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5","openshift-cluster-node-tuning-operator/tuned-rqjvq","openshift-dns/node-resolver-rxwbd","openshift-image-registry/node-ca-pf6hp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal","openshift-multus/multus-additional-cni-plugins-skm5h","openshift-multus/multus-whpvz","kube-system/global-pull-secret-syncer-8kq7q","kube-system/konnectivity-agent-q2wd4","openshift-multus/network-metrics-daemon-zpgj8"]
Jul 17 10:56:59.559985 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.556950    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.559985 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.558863    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.559985 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.559600    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 10:56:59.559985 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.559666    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 10:56:59.559985 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.559723    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g78kz\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.560551    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.561249    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.561476    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.561514    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x6s7k\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.561578    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.562261    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.563115    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4nrpq\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.563124    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.563435    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.563446    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.563471    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.563437    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.563790    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.564283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.564671    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.564747    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.564849    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.564883    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hkhm8\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.565841    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.566709    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.566742    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.566909    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fdx6n\""
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.567273    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:56:59.568002 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.567335    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:56:59.569438 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.568302    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 10:56:59.569438 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.568429    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 10:56:59.569438 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.568467    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 10:56:59.569438 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.568556    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87k2x\""
Jul 17 10:56:59.569438 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.568937    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.570294    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.570616    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.571370    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.571742    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.571795    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.571847    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vw8c9\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.571890    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.571900    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.572514    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.572581    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.573084    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.573084    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-psgt2\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.574159    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.575563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.575627    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.576497    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.576881    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.576912    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7pvpj\""
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582327    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p69mj\" (UniqueName: \"kubernetes.io/projected/589aee41-a820-4f03-adb2-60ac64238bee-kube-api-access-p69mj\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582352    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-var-lib-openvswitch\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582378    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582398    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-os-release\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.680048 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582439    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-sysconfig\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582465    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-cni-bin\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582494    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aefeda7-ce38-4020-8d0b-6ea66c2d3d36-tmp-dir\") pod \"node-resolver-rxwbd\" (UID: \"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36\") " pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-cnibin\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582561    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-var-lib-kubelet\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582578    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-kubelet\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582596    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-kubernetes\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582648    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-run\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582678    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/81258e2e-ceb5-445c-ac11-536fa9c6c163-iptables-alerter-script\") pod \"iptables-alerter-gb5wc\" (UID: \"81258e2e-ceb5-445c-ac11-536fa9c6c163\") " pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ece5b685-5a25-414a-8b9a-d6de919e4f6c-ovn-node-metrics-cert\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582790    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-etc-kubernetes\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-var-lib-kubelet\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582873    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-run-ovn\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.683593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582889    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ece5b685-5a25-414a-8b9a-d6de919e4f6c-env-overrides\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582909    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ece5b685-5a25-414a-8b9a-d6de919e4f6c-ovnkube-script-lib\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582952    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-hostroot\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.582981    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-sys\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583021    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583427    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ece5b685-5a25-414a-8b9a-d6de919e4f6c-ovnkube-config\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583506    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-modprobe-d\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583715    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-lib-modules\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583758    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kclf2\" (UniqueName: \"kubernetes.io/projected/ece5b685-5a25-414a-8b9a-d6de919e4f6c-kube-api-access-kclf2\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583799    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-socket-dir-parent\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583815    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-run-netns\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583835    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-sys-fs\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583933    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-sysctl-d\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.684503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.583985    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-systemd\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584028    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-registration-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584057    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-cni-binary-copy\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-node-log\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584122    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-system-cni-dir\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584171    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-cnibin\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584202    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-system-cni-dir\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-cni-dir\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-sysctl-conf\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584324    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-host\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-tuned\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584443    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-systemd-units\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584626    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-cni-binary-copy\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584651    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-run-k8s-cni-cncf-io\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.690368 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584684    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e-serviceca\") pod \"node-ca-pf6hp\" (UID: \"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e\") " pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584711    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584751    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-device-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584840    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81258e2e-ceb5-445c-ac11-536fa9c6c163-host-slash\") pod \"iptables-alerter-gb5wc\" (UID: \"81258e2e-ceb5-445c-ac11-536fa9c6c163\") " pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584875    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-run-netns\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584927    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-etc-openvswitch\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.584966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5dd1575-2e80-474d-b036-22c201bcc5db-tmp\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585010    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6aefeda7-ce38-4020-8d0b-6ea66c2d3d36-hosts-file\") pod \"node-resolver-rxwbd\" (UID: \"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36\") " pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585057    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d4x7\" (UniqueName: \"kubernetes.io/projected/6aefeda7-ce38-4020-8d0b-6ea66c2d3d36-kube-api-access-7d4x7\") pod \"node-resolver-rxwbd\" (UID: \"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36\") " pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585093    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-tuning-conf-dir\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585118    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-var-lib-cni-bin\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-slash\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585280    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-socket-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.691300 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585333    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-run-systemd\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585365    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585390    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sptph\" (UniqueName: \"kubernetes.io/projected/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-kube-api-access-sptph\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585421    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-var-lib-cni-multus\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585451    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-daemon-config\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-run-multus-certs\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e-host\") pod \"node-ca-pf6hp\" (UID: \"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e\") " pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585543    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrtbj\" (UniqueName: \"kubernetes.io/projected/81258e2e-ceb5-445c-ac11-536fa9c6c163-kube-api-access-xrtbj\") pod \"iptables-alerter-gb5wc\" (UID: \"81258e2e-ceb5-445c-ac11-536fa9c6c163\") " pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585568    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-cni-netd\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585597    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-conf-dir\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585630    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xcq2\" (UniqueName: \"kubernetes.io/projected/cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e-kube-api-access-7xcq2\") pod \"node-ca-pf6hp\" (UID: \"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e\") " pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585659    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-run-openvswitch\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585684    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s477\" (UniqueName: \"kubernetes.io/projected/f5dd1575-2e80-474d-b036-22c201bcc5db-kube-api-access-4s477\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585714    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-os-release\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.809615 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585767    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-log-socket\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585800    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf4zk\" (UniqueName: \"kubernetes.io/projected/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-kube-api-access-xf4zk\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.585833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-etc-selinux\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.609268    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f48h4"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.618028    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f48h4"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.670425    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686216    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-host\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-tuned\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686259    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-systemd-units\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686285    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-cni-binary-copy\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686311    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-run-k8s-cni-cncf-io\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686336    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e-serviceca\") pod \"node-ca-pf6hp\" (UID: \"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e\") " pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686359    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-device-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-systemd-units\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686336    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-host\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686413    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-run-k8s-cni-cncf-io\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.810951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686441    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81258e2e-ceb5-445c-ac11-536fa9c6c163-host-slash\") pod \"iptables-alerter-gb5wc\" (UID: \"81258e2e-ceb5-445c-ac11-536fa9c6c163\") " pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686471    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-run-netns\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686488    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-device-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686496    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81258e2e-ceb5-445c-ac11-536fa9c6c163-host-slash\") pod \"iptables-alerter-gb5wc\" (UID: \"81258e2e-ceb5-445c-ac11-536fa9c6c163\") " pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686501    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-etc-openvswitch\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686551    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5dd1575-2e80-474d-b036-22c201bcc5db-tmp\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-etc-openvswitch\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686559    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-run-netns\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686586    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6aefeda7-ce38-4020-8d0b-6ea66c2d3d36-hosts-file\") pod \"node-resolver-rxwbd\" (UID: \"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36\") " pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686607    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d4x7\" (UniqueName: \"kubernetes.io/projected/6aefeda7-ce38-4020-8d0b-6ea66c2d3d36-kube-api-access-7d4x7\") pod \"node-resolver-rxwbd\" (UID: \"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36\") " pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686660    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6aefeda7-ce38-4020-8d0b-6ea66c2d3d36-hosts-file\") pod \"node-resolver-rxwbd\" (UID: \"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36\") " pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-tuning-conf-dir\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686761    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-var-lib-cni-bin\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-slash\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.811835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686794    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686812    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-socket-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-run-systemd\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686861    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sptph\" (UniqueName: \"kubernetes.io/projected/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-kube-api-access-sptph\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686909    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-var-lib-cni-multus\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686912    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-run-systemd\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686925    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-daemon-config\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686956    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-run-multus-certs\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686972    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e-host\") pod \"node-ca-pf6hp\" (UID: \"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e\") " pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.686999    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrtbj\" (UniqueName: \"kubernetes.io/projected/81258e2e-ceb5-445c-ac11-536fa9c6c163-kube-api-access-xrtbj\") pod \"iptables-alerter-gb5wc\" (UID: \"81258e2e-ceb5-445c-ac11-536fa9c6c163\") " pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687015    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-cni-netd\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-conf-dir\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687081    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xcq2\" (UniqueName: \"kubernetes.io/projected/cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e-kube-api-access-7xcq2\") pod \"node-ca-pf6hp\" (UID: \"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e\") " pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.813577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687114    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-var-lib-cni-bin\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687114    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0758dbfc-ac21-4268-83e7-1cb5053415b8-kubelet-config\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687152    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-run-openvswitch\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687170    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s477\" (UniqueName: \"kubernetes.io/projected/f5dd1575-2e80-474d-b036-22c201bcc5db-kube-api-access-4s477\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687176    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-conf-dir\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687205    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-os-release\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687212    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-slash\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687221    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e-host\") pod \"node-ca-pf6hp\" (UID: \"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e\") " pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4df32018-f34a-4db9-b449-1c6dddd2d6db-konnectivity-ca\") pod \"konnectivity-agent-q2wd4\" (UID: \"4df32018-f34a-4db9-b449-1c6dddd2d6db\") " pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687264    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-log-socket\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687279    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-cni-netd\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687286    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4zk\" (UniqueName: \"kubernetes.io/projected/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-kube-api-access-xf4zk\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687327    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-etc-selinux\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687088    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-run-multus-certs\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.814482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687367    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p69mj\" (UniqueName: \"kubernetes.io/projected/589aee41-a820-4f03-adb2-60ac64238bee-kube-api-access-p69mj\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687410    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-var-lib-openvswitch\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687466    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-os-release\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687525    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-var-lib-cni-multus\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687541    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-etc-selinux\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-sysconfig\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687598    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-sysconfig\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687630    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-os-release\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687633    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-var-lib-openvswitch\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687657    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-cni-bin\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687678    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-log-socket\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687682    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-cni-bin\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687691    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687745    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aefeda7-ce38-4020-8d0b-6ea66c2d3d36-tmp-dir\") pod \"node-resolver-rxwbd\" (UID: \"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36\") " pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.815803 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687747    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-run-openvswitch\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687740    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-os-release\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687785    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-cnibin\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687819    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-var-lib-kubelet\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687830    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-cnibin\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687863    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-var-lib-kubelet\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-kubelet\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687885    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-kubernetes\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-run\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687913    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-kubelet\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687923    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/81258e2e-ceb5-445c-ac11-536fa9c6c163-iptables-alerter-script\") pod \"iptables-alerter-gb5wc\" (UID: \"81258e2e-ceb5-445c-ac11-536fa9c6c163\") " pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687939    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-kubernetes\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687946    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ece5b685-5a25-414a-8b9a-d6de919e4f6c-ovn-node-metrics-cert\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687964    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-etc-kubernetes\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.687979    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.816681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688031    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-run\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688071    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688090    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-etc-kubernetes\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688113    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6t7b\" (UniqueName: \"kubernetes.io/projected/4b9c0e03-0542-4a98-901e-fddc777badf8-kube-api-access-w6t7b\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688133    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-var-lib-kubelet\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688149    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-run-ovn\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ece5b685-5a25-414a-8b9a-d6de919e4f6c-env-overrides\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688206    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ece5b685-5a25-414a-8b9a-d6de919e4f6c-ovnkube-script-lib\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688209    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-var-lib-kubelet\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-hostroot\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688230    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-run-ovn\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688254    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688271    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-sys\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688291    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.818474 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ece5b685-5a25-414a-8b9a-d6de919e4f6c-ovnkube-config\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688339    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4df32018-f34a-4db9-b449-1c6dddd2d6db-agent-certs\") pod \"konnectivity-agent-q2wd4\" (UID: \"4df32018-f34a-4db9-b449-1c6dddd2d6db\") " pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688357    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-modprobe-d\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688373    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-lib-modules\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kclf2\" (UniqueName: \"kubernetes.io/projected/ece5b685-5a25-414a-8b9a-d6de919e4f6c-kube-api-access-kclf2\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688407    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-socket-dir-parent\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688425    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-run-netns\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688447    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-sys-fs\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688463    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-hostroot\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-sysctl-d\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688523    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-systemd\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688557    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-registration-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688565    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-sysctl-d\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.820697 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688594    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688628    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-cni-binary-copy\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688661    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-node-log\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688692    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-system-cni-dir\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-cnibin\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688890    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-system-cni-dir\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688920    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-cni-dir\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0758dbfc-ac21-4268-83e7-1cb5053415b8-dbus\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688996    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-sysctl-conf\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689051    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-tuning-conf-dir\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689085    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-socket-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.688406    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689142    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-sysctl-conf\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689206    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-systemd\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.821175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689256    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-modprobe-d\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689294    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-sys\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689316    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-cni-dir\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689336    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-system-cni-dir\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5dd1575-2e80-474d-b036-22c201bcc5db-lib-modules\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689265    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-socket-dir-parent\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689348    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-registration-dir\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689375    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-system-cni-dir\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689397    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-cnibin\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689405    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ece5b685-5a25-414a-8b9a-d6de919e4f6c-node-log\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689422    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/589aee41-a820-4f03-adb2-60ac64238bee-sys-fs\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.689621    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-host-run-netns\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.692225    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.692355    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.693062    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-cni-binary-copy\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.821574 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.693157    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e-serviceca\") pod \"node-ca-pf6hp\" (UID: \"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e\") " pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.693225    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ece5b685-5a25-414a-8b9a-d6de919e4f6c-ovnkube-script-lib\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.693263    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ece5b685-5a25-414a-8b9a-d6de919e4f6c-ovnkube-config\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.693377    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.693731    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/81258e2e-ceb5-445c-ac11-536fa9c6c163-iptables-alerter-script\") pod \"iptables-alerter-gb5wc\" (UID: \"81258e2e-ceb5-445c-ac11-536fa9c6c163\") " pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.693874    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-multus-daemon-config\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.694093    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aefeda7-ce38-4020-8d0b-6ea66c2d3d36-tmp-dir\") pod \"node-resolver-rxwbd\" (UID: \"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36\") " pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.694163    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-cni-binary-copy\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.694189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ece5b685-5a25-414a-8b9a-d6de919e4f6c-env-overrides\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.695698    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5dd1575-2e80-474d-b036-22c201bcc5db-tmp\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.696115    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f5dd1575-2e80-474d-b036-22c201bcc5db-etc-tuned\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.696459    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.696483    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.696497    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xnw2h for pod openshift-network-diagnostics/network-check-target-m86vn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:56:59.822047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.696455    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ece5b685-5a25-414a-8b9a-d6de919e4f6c-ovn-node-metrics-cert\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.696597    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h podName:acd844fd-aed6-46c2-bd12-43fd4e7298fd nodeName:}" failed. No retries permitted until 2026-07-17 10:57:00.196560317 +0000 UTC m=+3.053644259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xnw2h" (UniqueName: "kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h") pod "network-check-target-m86vn" (UID: "acd844fd-aed6-46c2-bd12-43fd4e7298fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.698540    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s477\" (UniqueName: \"kubernetes.io/projected/f5dd1575-2e80-474d-b036-22c201bcc5db-kube-api-access-4s477\") pod \"tuned-rqjvq\" (UID: \"f5dd1575-2e80-474d-b036-22c201bcc5db\") " pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.698543    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d4x7\" (UniqueName: \"kubernetes.io/projected/6aefeda7-ce38-4020-8d0b-6ea66c2d3d36-kube-api-access-7d4x7\") pod \"node-resolver-rxwbd\" (UID: \"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36\") " pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.698722    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sptph\" (UniqueName: \"kubernetes.io/projected/403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561-kube-api-access-sptph\") pod \"multus-additional-cni-plugins-skm5h\" (UID: \"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561\") " pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.698939    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf4zk\" (UniqueName: \"kubernetes.io/projected/f7c2448d-5624-4eba-9f50-cd3777a9c5ba-kube-api-access-xf4zk\") pod \"multus-whpvz\" (UID: \"f7c2448d-5624-4eba-9f50-cd3777a9c5ba\") " pod="openshift-multus/multus-whpvz"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.699122    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p69mj\" (UniqueName: \"kubernetes.io/projected/589aee41-a820-4f03-adb2-60ac64238bee-kube-api-access-p69mj\") pod \"aws-ebs-csi-driver-node-qwjr5\" (UID: \"589aee41-a820-4f03-adb2-60ac64238bee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.699292    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kclf2\" (UniqueName: \"kubernetes.io/projected/ece5b685-5a25-414a-8b9a-d6de919e4f6c-kube-api-access-kclf2\") pod \"ovnkube-node-sl2kk\" (UID: \"ece5b685-5a25-414a-8b9a-d6de919e4f6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.699770    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xcq2\" (UniqueName: \"kubernetes.io/projected/cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e-kube-api-access-7xcq2\") pod \"node-ca-pf6hp\" (UID: \"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e\") " pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.699914    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrtbj\" (UniqueName: \"kubernetes.io/projected/81258e2e-ceb5-445c-ac11-536fa9c6c163-kube-api-access-xrtbj\") pod \"iptables-alerter-gb5wc\" (UID: \"81258e2e-ceb5-445c-ac11-536fa9c6c163\") " pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.754667    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.789435    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0758dbfc-ac21-4268-83e7-1cb5053415b8-kubelet-config\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.789463    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4df32018-f34a-4db9-b449-1c6dddd2d6db-konnectivity-ca\") pod \"konnectivity-agent-q2wd4\" (UID: \"4df32018-f34a-4db9-b449-1c6dddd2d6db\") " pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:56:59.822437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.789485    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6t7b\" (UniqueName: \"kubernetes.io/projected/4b9c0e03-0542-4a98-901e-fddc777badf8-kube-api-access-w6t7b\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.789513    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.789558    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0758dbfc-ac21-4268-83e7-1cb5053415b8-kubelet-config\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.789598    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.789645    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret podName:0758dbfc-ac21-4268-83e7-1cb5053415b8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:00.289629955 +0000 UTC m=+3.146713880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret") pod "global-pull-secret-syncer-8kq7q" (UID: "0758dbfc-ac21-4268-83e7-1cb5053415b8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.789668    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4df32018-f34a-4db9-b449-1c6dddd2d6db-agent-certs\") pod \"konnectivity-agent-q2wd4\" (UID: \"4df32018-f34a-4db9-b449-1c6dddd2d6db\") " pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.789729    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.789761    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0758dbfc-ac21-4268-83e7-1cb5053415b8-dbus\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.789843    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0758dbfc-ac21-4268-83e7-1cb5053415b8-dbus\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.789882    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:56:59.789991    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs podName:4b9c0e03-0542-4a98-901e-fddc777badf8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:00.289971048 +0000 UTC m=+3.147054976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs") pod "network-metrics-daemon-zpgj8" (UID: "4b9c0e03-0542-4a98-901e-fddc777badf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.790109    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4df32018-f34a-4db9-b449-1c6dddd2d6db-konnectivity-ca\") pod \"konnectivity-agent-q2wd4\" (UID: \"4df32018-f34a-4db9-b449-1c6dddd2d6db\") " pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.791886    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4df32018-f34a-4db9-b449-1c6dddd2d6db-agent-certs\") pod \"konnectivity-agent-q2wd4\" (UID: \"4df32018-f34a-4db9-b449-1c6dddd2d6db\") " pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:56:59.822828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.799072    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6t7b\" (UniqueName: \"kubernetes.io/projected/4b9c0e03-0542-4a98-901e-fddc777badf8-kube-api-access-w6t7b\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:56:59.954651 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.954630    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rqjvq"
Jul 17 10:56:59.972396 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.972368    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gb5wc"
Jul 17 10:56:59.979552 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.979530    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:56:59.992242 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:56:59.992224    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pf6hp"
Jul 17 10:57:00.008245 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.008227    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rxwbd"
Jul 17 10:57:00.021273 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.021256    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5"
Jul 17 10:57:00.037453 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.037430    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-skm5h"
Jul 17 10:57:00.059159 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.059134    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-whpvz"
Jul 17 10:57:00.061136 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.061108    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod403c6ed9_8bd8_4b0a_bce9_e4d2ea66d561.slice/crio-2734ffcdd8a49f66997fea7c4fdae5076cc4177eaeab15ac592a360533522c7a WatchSource:0}: Error finding container 2734ffcdd8a49f66997fea7c4fdae5076cc4177eaeab15ac592a360533522c7a: Status 404 returned error can't find the container with id 2734ffcdd8a49f66997fea7c4fdae5076cc4177eaeab15ac592a360533522c7a
Jul 17 10:57:00.062143 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.062120    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47af943b4b7e55afd17e31a4e269c392.slice/crio-cf49b9821cc041bc30305e2b4041f6fec8b788375950225614a4447654a6e946 WatchSource:0}: Error finding container cf49b9821cc041bc30305e2b4041f6fec8b788375950225614a4447654a6e946: Status 404 returned error can't find the container with id cf49b9821cc041bc30305e2b4041f6fec8b788375950225614a4447654a6e946
Jul 17 10:57:00.062555 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.062535    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105a462c32e44b70fe7bbcf79e028e08.slice/crio-c764e6c638931c852a0e2cffa84494e09b1f8e5f048e97a7d716c89356f9074a WatchSource:0}: Error finding container c764e6c638931c852a0e2cffa84494e09b1f8e5f048e97a7d716c89356f9074a: Status 404 returned error can't find the container with id c764e6c638931c852a0e2cffa84494e09b1f8e5f048e97a7d716c89356f9074a
Jul 17 10:57:00.067264 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.067243    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c2448d_5624_4eba_9f50_cd3777a9c5ba.slice/crio-a99df09aba8baadcbfa388fde90c4cf5d58a4bde014850461aa9a91496268898 WatchSource:0}: Error finding container a99df09aba8baadcbfa388fde90c4cf5d58a4bde014850461aa9a91496268898: Status 404 returned error can't find the container with id a99df09aba8baadcbfa388fde90c4cf5d58a4bde014850461aa9a91496268898
Jul 17 10:57:00.068104 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.068080    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 10:57:00.073034 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.073017    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:57:00.080740 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.080717    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df32018_f34a_4db9_b449_1c6dddd2d6db.slice/crio-39b7725d71295b3cbeca7e5c7e38de8c780c4f147ddc97496fefc8e96eb3c61e WatchSource:0}: Error finding container 39b7725d71295b3cbeca7e5c7e38de8c780c4f147ddc97496fefc8e96eb3c61e: Status 404 returned error can't find the container with id 39b7725d71295b3cbeca7e5c7e38de8c780c4f147ddc97496fefc8e96eb3c61e
Jul 17 10:57:00.255238 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.255212    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5dd1575_2e80_474d_b036_22c201bcc5db.slice/crio-a1fe861c9719d17a84e8feafff4b3f8df4b93024fa536232bdcf2f1afd16d9fb WatchSource:0}: Error finding container a1fe861c9719d17a84e8feafff4b3f8df4b93024fa536232bdcf2f1afd16d9fb: Status 404 returned error can't find the container with id a1fe861c9719d17a84e8feafff4b3f8df4b93024fa536232bdcf2f1afd16d9fb
Jul 17 10:57:00.293608 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.293584    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:00.293762 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.293615    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:00.293762 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.293634    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:00.293762 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:00.293730    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:00.293762 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:00.293751    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:00.293918 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:00.293769    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 10:57:00.293918 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:00.293783    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret podName:0758dbfc-ac21-4268-83e7-1cb5053415b8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:01.29376662 +0000 UTC m=+4.150850543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret") pod "global-pull-secret-syncer-8kq7q" (UID: "0758dbfc-ac21-4268-83e7-1cb5053415b8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:00.293918 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:00.293785    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 10:57:00.293918 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:00.293796    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs podName:4b9c0e03-0542-4a98-901e-fddc777badf8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:01.293790524 +0000 UTC m=+4.150874447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs") pod "network-metrics-daemon-zpgj8" (UID: "4b9c0e03-0542-4a98-901e-fddc777badf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:00.293918 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:00.293799    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xnw2h for pod openshift-network-diagnostics/network-check-target-m86vn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:00.293918 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:00.293840    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h podName:acd844fd-aed6-46c2-bd12-43fd4e7298fd nodeName:}" failed. No retries permitted until 2026-07-17 10:57:01.293824909 +0000 UTC m=+4.150908844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnw2h" (UniqueName: "kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h") pod "network-check-target-m86vn" (UID: "acd844fd-aed6-46c2-bd12-43fd4e7298fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:00.344744 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.344700    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfea99fc_e552_4e24_a93b_8bfe7a6ddb5e.slice/crio-62b49b29cae53097cdf27fd7e740e678f9bedb6d6e535ad8ae13b344239822bd WatchSource:0}: Error finding container 62b49b29cae53097cdf27fd7e740e678f9bedb6d6e535ad8ae13b344239822bd: Status 404 returned error can't find the container with id 62b49b29cae53097cdf27fd7e740e678f9bedb6d6e535ad8ae13b344239822bd
Jul 17 10:57:00.364590 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.364565    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81258e2e_ceb5_445c_ac11_536fa9c6c163.slice/crio-aec36ca23012d5c0b9b3b8e644e56b5c01380784323153378b009f39b7482657 WatchSource:0}: Error finding container aec36ca23012d5c0b9b3b8e644e56b5c01380784323153378b009f39b7482657: Status 404 returned error can't find the container with id aec36ca23012d5c0b9b3b8e644e56b5c01380784323153378b009f39b7482657
Jul 17 10:57:00.393102 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.393074    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aefeda7_ce38_4020_8d0b_6ea66c2d3d36.slice/crio-8c28b73f2457912b48ca18f21a7b951ea43ed94b30be7d759c77706337ae08a2 WatchSource:0}: Error finding container 8c28b73f2457912b48ca18f21a7b951ea43ed94b30be7d759c77706337ae08a2: Status 404 returned error can't find the container with id 8c28b73f2457912b48ca18f21a7b951ea43ed94b30be7d759c77706337ae08a2
Jul 17 10:57:00.398382 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.398356    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589aee41_a820_4f03_adb2_60ac64238bee.slice/crio-c7511c268269c38c2b4c2cbe1406b3f7de6d2f1d5aa5a437d6c7a806b1d5a880 WatchSource:0}: Error finding container c7511c268269c38c2b4c2cbe1406b3f7de6d2f1d5aa5a437d6c7a806b1d5a880: Status 404 returned error can't find the container with id c7511c268269c38c2b4c2cbe1406b3f7de6d2f1d5aa5a437d6c7a806b1d5a880
Jul 17 10:57:00.468372 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:00.468341    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece5b685_5a25_414a_8b9a_d6de919e4f6c.slice/crio-8f6595bd21a78d9218fd86727b0263604f83cca2cb34fdf2b9ae90048da3abaf WatchSource:0}: Error finding container 8f6595bd21a78d9218fd86727b0263604f83cca2cb34fdf2b9ae90048da3abaf: Status 404 returned error can't find the container with id 8f6595bd21a78d9218fd86727b0263604f83cca2cb34fdf2b9ae90048da3abaf
Jul 17 10:57:00.520806 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.520731    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 10:57:00.620482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.620432    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 10:51:59 +0000 UTC" deadline="2028-03-13 12:10:02.105858526 +0000 UTC"
Jul 17 10:57:00.620482 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.620477    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14521h13m1.485384686s"
Jul 17 10:57:00.673578 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.673489    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerStarted","Data":"8f6595bd21a78d9218fd86727b0263604f83cca2cb34fdf2b9ae90048da3abaf"}
Jul 17 10:57:00.674730 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.674687    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5" event={"ID":"589aee41-a820-4f03-adb2-60ac64238bee","Type":"ContainerStarted","Data":"c7511c268269c38c2b4c2cbe1406b3f7de6d2f1d5aa5a437d6c7a806b1d5a880"}
Jul 17 10:57:00.675940 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.675914    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rxwbd" event={"ID":"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36","Type":"ContainerStarted","Data":"8c28b73f2457912b48ca18f21a7b951ea43ed94b30be7d759c77706337ae08a2"}
Jul 17 10:57:00.677681 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.677653    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gb5wc" event={"ID":"81258e2e-ceb5-445c-ac11-536fa9c6c163","Type":"ContainerStarted","Data":"aec36ca23012d5c0b9b3b8e644e56b5c01380784323153378b009f39b7482657"}
Jul 17 10:57:00.678861 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.678840    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pf6hp" event={"ID":"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e","Type":"ContainerStarted","Data":"62b49b29cae53097cdf27fd7e740e678f9bedb6d6e535ad8ae13b344239822bd"}
Jul 17 10:57:00.680042 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.680018    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q2wd4" event={"ID":"4df32018-f34a-4db9-b449-1c6dddd2d6db","Type":"ContainerStarted","Data":"39b7725d71295b3cbeca7e5c7e38de8c780c4f147ddc97496fefc8e96eb3c61e"}
Jul 17 10:57:00.681281 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.681256    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-skm5h" event={"ID":"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561","Type":"ContainerStarted","Data":"2734ffcdd8a49f66997fea7c4fdae5076cc4177eaeab15ac592a360533522c7a"}
Jul 17 10:57:00.682390 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.682365    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rqjvq" event={"ID":"f5dd1575-2e80-474d-b036-22c201bcc5db","Type":"ContainerStarted","Data":"a1fe861c9719d17a84e8feafff4b3f8df4b93024fa536232bdcf2f1afd16d9fb"}
Jul 17 10:57:00.683607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.683586    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-whpvz" event={"ID":"f7c2448d-5624-4eba-9f50-cd3777a9c5ba","Type":"ContainerStarted","Data":"a99df09aba8baadcbfa388fde90c4cf5d58a4bde014850461aa9a91496268898"}
Jul 17 10:57:00.684868 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.684847    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal" event={"ID":"47af943b4b7e55afd17e31a4e269c392","Type":"ContainerStarted","Data":"cf49b9821cc041bc30305e2b4041f6fec8b788375950225614a4447654a6e946"}
Jul 17 10:57:00.685993 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:00.685969    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal" event={"ID":"105a462c32e44b70fe7bbcf79e028e08","Type":"ContainerStarted","Data":"c764e6c638931c852a0e2cffa84494e09b1f8e5f048e97a7d716c89356f9074a"}
Jul 17 10:57:01.304202 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:01.303933    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:01.304202 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:01.304022    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:01.304202 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:01.304065    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:01.304202 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.304205    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 10:57:01.304616 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.304226    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 10:57:01.304616 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.304238    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xnw2h for pod openshift-network-diagnostics/network-check-target-m86vn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:01.304616 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.304303    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h podName:acd844fd-aed6-46c2-bd12-43fd4e7298fd nodeName:}" failed. No retries permitted until 2026-07-17 10:57:03.30427815 +0000 UTC m=+6.161362090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnw2h" (UniqueName: "kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h") pod "network-check-target-m86vn" (UID: "acd844fd-aed6-46c2-bd12-43fd4e7298fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:01.304818 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.304718    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:01.304818 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.304771    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs podName:4b9c0e03-0542-4a98-901e-fddc777badf8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:03.304755741 +0000 UTC m=+6.161839680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs") pod "network-metrics-daemon-zpgj8" (UID: "4b9c0e03-0542-4a98-901e-fddc777badf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:01.304930 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.304826    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:01.304930 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.304863    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret podName:0758dbfc-ac21-4268-83e7-1cb5053415b8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:03.304849782 +0000 UTC m=+6.161933719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret") pod "global-pull-secret-syncer-8kq7q" (UID: "0758dbfc-ac21-4268-83e7-1cb5053415b8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:01.621420 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:01.621237    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 10:51:59 +0000 UTC" deadline="2028-01-25 23:45:36.740862352 +0000 UTC"
Jul 17 10:57:01.621420 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:01.621325    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13380h48m35.119586975s"
Jul 17 10:57:01.655634 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:01.654531    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:01.655634 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.654651    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:01.655634 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:01.655077    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:01.655634 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.655176    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:01.655634 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:01.655243    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:01.655634 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:01.655332    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:03.322963 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:03.322883    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:03.322963 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:03.322944    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:03.323578 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:03.322972    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:03.323578 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.323118    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:03.323578 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.323194    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs podName:4b9c0e03-0542-4a98-901e-fddc777badf8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:07.32316323 +0000 UTC m=+10.180247175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs") pod "network-metrics-daemon-zpgj8" (UID: "4b9c0e03-0542-4a98-901e-fddc777badf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:03.323787 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.323579    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:03.323787 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.323633    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret podName:0758dbfc-ac21-4268-83e7-1cb5053415b8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:07.323614941 +0000 UTC m=+10.180698865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret") pod "global-pull-secret-syncer-8kq7q" (UID: "0758dbfc-ac21-4268-83e7-1cb5053415b8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:03.323787 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.323735    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 10:57:03.323787 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.323755    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 10:57:03.323787 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.323768    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xnw2h for pod openshift-network-diagnostics/network-check-target-m86vn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:03.324024 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.323809    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h podName:acd844fd-aed6-46c2-bd12-43fd4e7298fd nodeName:}" failed. No retries permitted until 2026-07-17 10:57:07.323797079 +0000 UTC m=+10.180881003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnw2h" (UniqueName: "kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h") pod "network-check-target-m86vn" (UID: "acd844fd-aed6-46c2-bd12-43fd4e7298fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:03.654049 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:03.653975    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:03.654209 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.654099    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:03.654478 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:03.654458    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:03.654574 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.654555    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:03.654641 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:03.654611    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:03.654754 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:03.654718    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:05.655658 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:05.655526    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:05.655658 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:05.655575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:05.656174 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:05.655652    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:05.656174 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:05.655526    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:05.656174 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:05.655778    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:05.656174 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:05.655851    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:07.358023    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:07.358079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:07.358110    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.358272    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.358334    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs podName:4b9c0e03-0542-4a98-901e-fddc777badf8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:15.35831359 +0000 UTC m=+18.215397537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs") pod "network-metrics-daemon-zpgj8" (UID: "4b9c0e03-0542-4a98-901e-fddc777badf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.358565    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.358584    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.358596    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xnw2h for pod openshift-network-diagnostics/network-check-target-m86vn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.358640    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h podName:acd844fd-aed6-46c2-bd12-43fd4e7298fd nodeName:}" failed. No retries permitted until 2026-07-17 10:57:15.358626013 +0000 UTC m=+18.215709941 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnw2h" (UniqueName: "kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h") pod "network-check-target-m86vn" (UID: "acd844fd-aed6-46c2-bd12-43fd4e7298fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.358693    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:07.358738 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.358758    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret podName:0758dbfc-ac21-4268-83e7-1cb5053415b8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:15.35874559 +0000 UTC m=+18.215829525 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret") pod "global-pull-secret-syncer-8kq7q" (UID: "0758dbfc-ac21-4268-83e7-1cb5053415b8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:07.653060 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:07.652978    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:07.653218 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:07.653142    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:07.653218 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.653153    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:07.653218 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:07.653168    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:07.653392 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.653242    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:07.653392 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:07.653321    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:09.655643 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:09.655596    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:09.656100 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:09.655747    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:09.656186 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:09.656131    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:09.656238 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:09.656218    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:09.656291 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:09.656271    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:09.656391 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:09.656350    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:11.654903 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:11.654868    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:11.655358 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:11.654987    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:11.655358 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:11.654992    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:11.655358 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:11.655099    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:11.655358 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:11.654986    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:11.655358 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:11.655188    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:13.651660 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:13.651623    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:13.652140 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:13.651641    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:13.652140 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:13.651762    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:13.652140 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:13.651768    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:13.652140 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:13.651874    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:13.652140 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:13.651972    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:15.414478 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:15.414443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:15.414489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.414586    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:15.414599    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.414596    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.414684    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.414689    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.414722    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xnw2h for pod openshift-network-diagnostics/network-check-target-m86vn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.414658    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret podName:0758dbfc-ac21-4268-83e7-1cb5053415b8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:31.414637253 +0000 UTC m=+34.271721177 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret") pod "global-pull-secret-syncer-8kq7q" (UID: "0758dbfc-ac21-4268-83e7-1cb5053415b8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.414772    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs podName:4b9c0e03-0542-4a98-901e-fddc777badf8 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:31.414753272 +0000 UTC m=+34.271837195 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs") pod "network-metrics-daemon-zpgj8" (UID: "4b9c0e03-0542-4a98-901e-fddc777badf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:15.414979 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.414801    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h podName:acd844fd-aed6-46c2-bd12-43fd4e7298fd nodeName:}" failed. No retries permitted until 2026-07-17 10:57:31.414782372 +0000 UTC m=+34.271866295 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnw2h" (UniqueName: "kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h") pod "network-check-target-m86vn" (UID: "acd844fd-aed6-46c2-bd12-43fd4e7298fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:15.651550 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:15.651510    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:15.651739 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:15.651519    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:15.651739 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.651631    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:15.651739 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:15.651519    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:15.651904 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.651767    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:15.651904 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:15.651792    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:17.654165 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:17.654139    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:17.654578 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:17.654139    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:17.654578 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:17.654233    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:17.654578 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:17.654150    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:17.654578 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:17.654307    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:17.654578 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:17.654407    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:19.652203 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:19.652168    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:19.652672 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:19.652169    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:19.652672 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:19.652300    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:19.652672 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:19.652395    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:19.652672 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:19.652185    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:19.652672 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:19.652498    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:20.744799 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.744579    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rqjvq" event={"ID":"f5dd1575-2e80-474d-b036-22c201bcc5db","Type":"ContainerStarted","Data":"6b8df8ecebb20f6728c8973d2a847b825880b0c9484c264f5b0d1b625bbf6c41"}
Jul 17 10:57:20.746517 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.746493    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-whpvz" event={"ID":"f7c2448d-5624-4eba-9f50-cd3777a9c5ba","Type":"ContainerStarted","Data":"305aeef9271450d5c395e184a0c6f5a08c61513bfde9e6e4acbc0d4d8d1bff89"}
Jul 17 10:57:20.747867 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.747846    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal" event={"ID":"47af943b4b7e55afd17e31a4e269c392","Type":"ContainerStarted","Data":"d1cb8427815bef7ca21d531c61f0fe1efe5ad168ff537b857ee4e7277ad41fa2"}
Jul 17 10:57:20.749165 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.749144    2566 generic.go:356] "Generic (PLEG): container finished" podID="105a462c32e44b70fe7bbcf79e028e08" containerID="5bfa2b4e990a94e719b3cd204d013b3fde37d6218e2d379cb999b5c8a190d7a6" exitCode=0
Jul 17 10:57:20.749279 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.749222    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal" event={"ID":"105a462c32e44b70fe7bbcf79e028e08","Type":"ContainerDied","Data":"5bfa2b4e990a94e719b3cd204d013b3fde37d6218e2d379cb999b5c8a190d7a6"}
Jul 17 10:57:20.751879 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.751863    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 10:57:20.752177 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.752157    2566 generic.go:356] "Generic (PLEG): container finished" podID="ece5b685-5a25-414a-8b9a-d6de919e4f6c" containerID="998b1fe2c10cb4c9bf8d4c1b1b4266b423721d9b547e401bfe88acf413390531" exitCode=1
Jul 17 10:57:20.752244 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.752226    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerStarted","Data":"3e193a4123192a4f532378e8452a9f63c65f4a0983af7de734eb0d8b43781a75"}
Jul 17 10:57:20.752324 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.752304    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerStarted","Data":"aff19ca2016634df040fed9055435898c0020828cc486a351c7ce0584913b33f"}
Jul 17 10:57:20.752414 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.752365    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerStarted","Data":"78b2a7e9c1594874d140337d065bf644c5f0b7b92c1bf49f30d821d2ee626dc1"}
Jul 17 10:57:20.752414 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.752409    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerStarted","Data":"d444bf0bdbfdc8253f1c788a38b673784622cd59bc9ecedb7c43b78d55ad581f"}
Jul 17 10:57:20.752492 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.752447    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerDied","Data":"998b1fe2c10cb4c9bf8d4c1b1b4266b423721d9b547e401bfe88acf413390531"}
Jul 17 10:57:20.752492 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.752485    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerStarted","Data":"93db0d1c873f74ff6974d9fc0f8ca755101ec1a13798a55bfd4cefa909c1cc94"}
Jul 17 10:57:20.753525 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.753507    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5" event={"ID":"589aee41-a820-4f03-adb2-60ac64238bee","Type":"ContainerStarted","Data":"eaa93625282dc84a082bb00c4eef3b81f43583d38756910dbc8001e60a3db702"}
Jul 17 10:57:20.754801 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.754779    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rxwbd" event={"ID":"6aefeda7-ce38-4020-8d0b-6ea66c2d3d36","Type":"ContainerStarted","Data":"3f48dec5e69105ae8b767df300a8040964f89f7b193db6a699469b044a042489"}
Jul 17 10:57:20.756044 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.756017    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pf6hp" event={"ID":"cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e","Type":"ContainerStarted","Data":"5c5e5c6b777f85ab85056ee28ca112292a689fe5d9f7195d4b7c2593bc35f83c"}
Jul 17 10:57:20.757173 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.757154    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q2wd4" event={"ID":"4df32018-f34a-4db9-b449-1c6dddd2d6db","Type":"ContainerStarted","Data":"b5d6a8220c0f1924bcd39f611c4b079f15be49e2df164c57f558643fce7cf5a7"}
Jul 17 10:57:20.758493 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.758471    2566 generic.go:356] "Generic (PLEG): container finished" podID="403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561" containerID="611a92b597238c1673eed71fa9048671afb64a113a0014a7c56d1d1d57fafec3" exitCode=0
Jul 17 10:57:20.758572 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.758500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-skm5h" event={"ID":"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561","Type":"ContainerDied","Data":"611a92b597238c1673eed71fa9048671afb64a113a0014a7c56d1d1d57fafec3"}
Jul 17 10:57:20.765376 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.765338    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rqjvq" podStartSLOduration=4.007147993 podStartE2EDuration="23.765326082s" totalImagesPullingTime="19.758178089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:20.764543457 +0000 UTC m=+23.621627402" watchObservedRunningTime="2026-07-17 10:57:20.765326082 +0000 UTC m=+23.622410017"
Jul 17 10:57:20.799002 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.798963    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rxwbd" podStartSLOduration=3.180654844 podStartE2EDuration="22.798949802s" totalImagesPullingTime="19.618294958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:20.798520783 +0000 UTC m=+23.655604729" watchObservedRunningTime="2026-07-17 10:57:20.798949802 +0000 UTC m=+23.656033747"
Jul 17 10:57:20.799115 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.799043    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pf6hp" podStartSLOduration=4.134962627 podStartE2EDuration="23.79903426s" totalImagesPullingTime="19.664071633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:20.780510521 +0000 UTC m=+23.637594465" watchObservedRunningTime="2026-07-17 10:57:20.79903426 +0000 UTC m=+23.656118212"
Jul 17 10:57:20.865845 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.865799    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-whpvz" podStartSLOduration=2.859698833 podStartE2EDuration="22.865780878s" totalImagesPullingTime="20.006082045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:20.865406938 +0000 UTC m=+23.722490883" watchObservedRunningTime="2026-07-17 10:57:20.865780878 +0000 UTC m=+23.722864824"
Jul 17 10:57:20.894258 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:20.894104    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q2wd4" podStartSLOduration=2.962517695 podStartE2EDuration="22.894090958s" totalImagesPullingTime="19.931573263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:20.893655697 +0000 UTC m=+23.750739641" watchObservedRunningTime="2026-07-17 10:57:20.894090958 +0000 UTC m=+23.751174903"
Jul 17 10:57:21.651409 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:21.651375    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:21.651787 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:21.651480    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:21.651787 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:21.651516    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:21.651787 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:21.651598    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:21.651787 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:21.651617    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:21.651787 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:21.651685    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:21.761979 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:21.761939    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gb5wc" event={"ID":"81258e2e-ceb5-445c-ac11-536fa9c6c163","Type":"ContainerStarted","Data":"8bb02926ff79ed9b4d17b35ad439c9120cd770a0be2a1654cea78437bff25c4d"}
Jul 17 10:57:21.764020 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:21.763991    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal" event={"ID":"105a462c32e44b70fe7bbcf79e028e08","Type":"ContainerStarted","Data":"f1c5a92eaf75cef34a06c55c73d4b20282227098e3e041e6e42bd0a20729f70f"}
Jul 17 10:57:21.780359 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:21.780304    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gb5wc" podStartSLOduration=5.132548022 podStartE2EDuration="24.78029069s" totalImagesPullingTime="19.647742668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:21.780131251 +0000 UTC m=+24.637215195" watchObservedRunningTime="2026-07-17 10:57:21.78029069 +0000 UTC m=+24.637374646"
Jul 17 10:57:21.780530 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:21.780509    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal" podStartSLOduration=23.780503976 podStartE2EDuration="23.780503976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:20.946828488 +0000 UTC m=+23.803912434" watchObservedRunningTime="2026-07-17 10:57:21.780503976 +0000 UTC m=+24.637587920"
Jul 17 10:57:21.804180 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:21.804132    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal" podStartSLOduration=23.804119039 podStartE2EDuration="23.804119039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:21.803034947 +0000 UTC m=+24.660118893" watchObservedRunningTime="2026-07-17 10:57:21.804119039 +0000 UTC m=+24.661202983"
Jul 17 10:57:22.092259 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:22.092235    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 10:57:22.670806 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:22.670687    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T10:57:22.092255263Z","UUID":"f3c31b26-b87c-4196-aff5-641a4a240d35","Handler":null,"Name":"","Endpoint":""}
Jul 17 10:57:22.672318 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:22.672297    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 10:57:22.672457 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:22.672340    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 10:57:22.769175 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:22.769144    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5" event={"ID":"589aee41-a820-4f03-adb2-60ac64238bee","Type":"ContainerStarted","Data":"7b639b5be57f9e14de096fe07a00783dc6d6657cabf5e25e9dcec5f66d502879"}
Jul 17 10:57:23.543595 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.543566    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:57:23.544845 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.544816    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:57:23.651951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.651920    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:23.652096 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.651920    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:23.652096 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:23.652035    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:23.652165 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:23.652139    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:23.652221 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.652206    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:23.652327 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:23.652310    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:23.775077 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.775007    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 10:57:23.775521 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.775425    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerStarted","Data":"529f82777c814367c45f41ae259812fa50e6f14239b55b47b1a958352ae04123"}
Jul 17 10:57:23.777693 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.777668    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5" event={"ID":"589aee41-a820-4f03-adb2-60ac64238bee","Type":"ContainerStarted","Data":"be29145332ac60fdb86eb6f88b15e3ce032e29b0eb5b36613ccc7cc4b9d25207"}
Jul 17 10:57:23.778065 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.778021    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:57:23.778503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.778484    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q2wd4"
Jul 17 10:57:23.796679 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:23.796634    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwjr5" podStartSLOduration=3.573640539 podStartE2EDuration="25.796620463s" totalImagesPullingTime="22.222979924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:23.796579658 +0000 UTC m=+26.653663602" watchObservedRunningTime="2026-07-17 10:57:23.796620463 +0000 UTC m=+26.653704409"
Jul 17 10:57:25.652257 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.652097    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:25.652845 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.652100    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:25.652845 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:25.652329    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:25.652845 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.652159    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:25.652845 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:25.652378    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:25.652845 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:25.652453    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:25.785529 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.785501    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 10:57:25.785930 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.785905    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerStarted","Data":"6f4142352595681e71cf5542a100083bdbfbd308506d69521708c25341d32d4f"}
Jul 17 10:57:25.786241 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.786215    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:57:25.786307 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.786300    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:57:25.786356 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.786338    2566 scope.go:117] "RemoveContainer" containerID="998b1fe2c10cb4c9bf8d4c1b1b4266b423721d9b547e401bfe88acf413390531"
Jul 17 10:57:25.787815 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.787787    2566 generic.go:356] "Generic (PLEG): container finished" podID="403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561" containerID="eb2dac7fb6ccf5ce1464dab08d336e92b96f7f4ffe187459f43dfb3f8761491a" exitCode=0
Jul 17 10:57:25.787924 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.787836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-skm5h" event={"ID":"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561","Type":"ContainerDied","Data":"eb2dac7fb6ccf5ce1464dab08d336e92b96f7f4ffe187459f43dfb3f8761491a"}
Jul 17 10:57:25.801222 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:25.801204    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:57:26.794143 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:26.794110    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 10:57:26.794759 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:26.794658    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" event={"ID":"ece5b685-5a25-414a-8b9a-d6de919e4f6c","Type":"ContainerStarted","Data":"534908a69032805eecad7b525375eede890f8990293ebb2674195bece6b03893"}
Jul 17 10:57:26.795950 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:26.795924    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:57:26.814467 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:26.814446    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:57:26.832361 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:26.832323    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk" podStartSLOduration=10.217766158 podStartE2EDuration="29.83231143s" totalImagesPullingTime="19.614545272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:26.830890542 +0000 UTC m=+29.687974487" watchObservedRunningTime="2026-07-17 10:57:26.83231143 +0000 UTC m=+29.689395375"
Jul 17 10:57:27.582425 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:27.582244    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8kq7q"]
Jul 17 10:57:27.582890 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:27.582499    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:27.582890 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:27.582581    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:27.590626 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:27.590604    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m86vn"]
Jul 17 10:57:27.590734 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:27.590687    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:27.590783 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:27.590769    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:27.628864 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:27.628835    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zpgj8"]
Jul 17 10:57:27.628968 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:27.628951    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:27.629051 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:27.629030    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:27.798850 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:27.798819    2566 generic.go:356] "Generic (PLEG): container finished" podID="403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561" containerID="4577db920a32c26b746af7fb301d6143e308ac160de41892a7ac50fbd07e0e41" exitCode=0
Jul 17 10:57:27.799400 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:27.798945    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-skm5h" event={"ID":"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561","Type":"ContainerDied","Data":"4577db920a32c26b746af7fb301d6143e308ac160de41892a7ac50fbd07e0e41"}
Jul 17 10:57:29.651688 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:29.651647    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:29.652049 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:29.651688    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:29.652049 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:29.651803    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:29.652049 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:29.651843    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:29.652049 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:29.651877    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:29.652049 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:29.651963    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:29.805228 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:29.805156    2566 generic.go:356] "Generic (PLEG): container finished" podID="403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561" containerID="cbf2dce52f2c3889ba9f97998fe8d4eae022b23151a0278bbad724be882ca651" exitCode=0
Jul 17 10:57:29.805228 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:29.805203    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-skm5h" event={"ID":"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561","Type":"ContainerDied","Data":"cbf2dce52f2c3889ba9f97998fe8d4eae022b23151a0278bbad724be882ca651"}
Jul 17 10:57:31.430151 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:31.430117    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:31.430169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:31.430201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.430283    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.430316    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.430326    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.430342    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.430352    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret podName:0758dbfc-ac21-4268-83e7-1cb5053415b8 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:03.430331452 +0000 UTC m=+66.287415391 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret") pod "global-pull-secret-syncer-8kq7q" (UID: "0758dbfc-ac21-4268-83e7-1cb5053415b8") : object "kube-system"/"original-pull-secret" not registered
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.430356    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xnw2h for pod openshift-network-diagnostics/network-check-target-m86vn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.430372    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs podName:4b9c0e03-0542-4a98-901e-fddc777badf8 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:03.430362504 +0000 UTC m=+66.287446427 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs") pod "network-metrics-daemon-zpgj8" (UID: "4b9c0e03-0542-4a98-901e-fddc777badf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 10:57:31.430640 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.430394    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h podName:acd844fd-aed6-46c2-bd12-43fd4e7298fd nodeName:}" failed. No retries permitted until 2026-07-17 10:58:03.430382574 +0000 UTC m=+66.287466512 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnw2h" (UniqueName: "kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h") pod "network-check-target-m86vn" (UID: "acd844fd-aed6-46c2-bd12-43fd4e7298fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 10:57:31.651977 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:31.651941    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:31.651977 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:31.651968    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:31.652203 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:31.651941    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:31.652203 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.652059    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kq7q" podUID="0758dbfc-ac21-4268-83e7-1cb5053415b8"
Jul 17 10:57:31.652203 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.652133    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zpgj8" podUID="4b9c0e03-0542-4a98-901e-fddc777badf8"
Jul 17 10:57:31.652203 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:31.652185    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m86vn" podUID="acd844fd-aed6-46c2-bd12-43fd4e7298fd"
Jul 17 10:57:32.940232 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:32.940159    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeReady"
Jul 17 10:57:32.940637 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:32.940323    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 10:57:32.980802 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:32.980774    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7576d667b4-h5t84"]
Jul 17 10:57:33.003515 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.003488    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7576d667b4-h5t84"]
Jul 17 10:57:33.003656 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.003525    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7hgrl"]
Jul 17 10:57:33.003656 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.003645    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.007149 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.007124    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 17 10:57:33.007303 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.007288    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 17 10:57:33.007303 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.007301    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 17 10:57:33.007457 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.007339    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nzx7f\""
Jul 17 10:57:33.013080 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.013058    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 17 10:57:33.016904 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.016882    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-47rdw"]
Jul 17 10:57:33.017033 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.017015    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.019616 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.019591    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 10:57:33.019616 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.019608    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 10:57:33.019806 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.019650    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w544b\""
Jul 17 10:57:33.034860 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.034843    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-47rdw"]
Jul 17 10:57:33.034975 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.034871    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7hgrl"]
Jul 17 10:57:33.034975 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.034952    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-47rdw"
Jul 17 10:57:33.037529 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.037511    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 10:57:33.037687 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.037666    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 10:57:33.038033 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.038007    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 10:57:33.038307 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.038291    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9lr9m\""
Jul 17 10:57:33.141955 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.141915    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/289bbdfa-ebd8-42af-be5d-95b7702a39f4-cert\") pod \"ingress-canary-47rdw\" (UID: \"289bbdfa-ebd8-42af-be5d-95b7702a39f4\") " pod="openshift-ingress-canary/ingress-canary-47rdw"
Jul 17 10:57:33.142142 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.141961    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-bound-sa-token\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.142142 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.141988    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcqj6\" (UniqueName: \"kubernetes.io/projected/679aabef-721d-436b-977a-87f5d0608638-kube-api-access-rcqj6\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.142142 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142102    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.142298 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142158    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr9f4\" (UniqueName: \"kubernetes.io/projected/289bbdfa-ebd8-42af-be5d-95b7702a39f4-kube-api-access-gr9f4\") pod \"ingress-canary-47rdw\" (UID: \"289bbdfa-ebd8-42af-be5d-95b7702a39f4\") " pod="openshift-ingress-canary/ingress-canary-47rdw"
Jul 17 10:57:33.142298 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142190    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qrfp\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-kube-api-access-2qrfp\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.142298 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142224    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679aabef-721d-436b-977a-87f5d0608638-tmp-dir\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.142440 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142295    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-image-registry-private-configuration\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.142440 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142325    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e973607b-b4cd-4b33-890f-b368d1e7e03a-ca-trust-extracted\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.142440 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142349    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-certificates\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.142440 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-trusted-ca\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.142440 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142401    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-installation-pull-secrets\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.142609 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/679aabef-721d-436b-977a-87f5d0608638-config-volume\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.142609 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.142486    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.243328 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243249    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/289bbdfa-ebd8-42af-be5d-95b7702a39f4-cert\") pod \"ingress-canary-47rdw\" (UID: \"289bbdfa-ebd8-42af-be5d-95b7702a39f4\") " pod="openshift-ingress-canary/ingress-canary-47rdw"
Jul 17 10:57:33.243328 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243290    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-bound-sa-token\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.243328 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243315    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcqj6\" (UniqueName: \"kubernetes.io/projected/679aabef-721d-436b-977a-87f5d0608638-kube-api-access-rcqj6\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243402    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr9f4\" (UniqueName: \"kubernetes.io/projected/289bbdfa-ebd8-42af-be5d-95b7702a39f4-kube-api-access-gr9f4\") pod \"ingress-canary-47rdw\" (UID: \"289bbdfa-ebd8-42af-be5d-95b7702a39f4\") " pod="openshift-ingress-canary/ingress-canary-47rdw"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243427    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qrfp\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-kube-api-access-2qrfp\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243451    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679aabef-721d-436b-977a-87f5d0608638-tmp-dir\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243482    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-image-registry-private-configuration\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e973607b-b4cd-4b33-890f-b368d1e7e03a-ca-trust-extracted\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243529    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-certificates\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243557    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-trusted-ca\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243586    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-installation-pull-secrets\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.243598    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243609    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/679aabef-721d-436b-977a-87f5d0608638-config-volume\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.243614 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.243617    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7576d667b4-h5t84: secret "image-registry-tls" not found
Jul 17 10:57:33.244203 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.243640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.244203 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.243685    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls podName:e973607b-b4cd-4b33-890f-b368d1e7e03a nodeName:}" failed. No retries permitted until 2026-07-17 10:57:33.743653564 +0000 UTC m=+36.600737512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls") pod "image-registry-7576d667b4-h5t84" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a") : secret "image-registry-tls" not found
Jul 17 10:57:33.244203 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.243740    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 10:57:33.244203 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.243789    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls podName:679aabef-721d-436b-977a-87f5d0608638 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:33.743772694 +0000 UTC m=+36.600856620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls") pod "dns-default-7hgrl" (UID: "679aabef-721d-436b-977a-87f5d0608638") : secret "dns-default-metrics-tls" not found
Jul 17 10:57:33.244203 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.244130    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679aabef-721d-436b-977a-87f5d0608638-tmp-dir\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.244541 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.244519    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/679aabef-721d-436b-977a-87f5d0608638-config-volume\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.244591 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.244543    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e973607b-b4cd-4b33-890f-b368d1e7e03a-ca-trust-extracted\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.244738 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.244700    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-certificates\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.245077 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.245032    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-trusted-ca\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.248207 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.248104    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-image-registry-private-configuration\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.248331 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.248131    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-installation-pull-secrets\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.248331 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.248189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/289bbdfa-ebd8-42af-be5d-95b7702a39f4-cert\") pod \"ingress-canary-47rdw\" (UID: \"289bbdfa-ebd8-42af-be5d-95b7702a39f4\") " pod="openshift-ingress-canary/ingress-canary-47rdw"
Jul 17 10:57:33.255400 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.255312    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcqj6\" (UniqueName: \"kubernetes.io/projected/679aabef-721d-436b-977a-87f5d0608638-kube-api-access-rcqj6\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.255665 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.255634    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr9f4\" (UniqueName: \"kubernetes.io/projected/289bbdfa-ebd8-42af-be5d-95b7702a39f4-kube-api-access-gr9f4\") pod \"ingress-canary-47rdw\" (UID: \"289bbdfa-ebd8-42af-be5d-95b7702a39f4\") " pod="openshift-ingress-canary/ingress-canary-47rdw"
Jul 17 10:57:33.256108 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.256084    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-bound-sa-token\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.256278 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.256243    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qrfp\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-kube-api-access-2qrfp\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.344637 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.344599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-47rdw"
Jul 17 10:57:33.651464 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.651428    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:57:33.651656 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.651428    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:57:33.651656 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.651589    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:57:33.654299 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.654277    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 10:57:33.654630 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.654407    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 10:57:33.654630 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.654413    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 10:57:33.654630 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.654475    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m4ccj\""
Jul 17 10:57:33.654630 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.654561    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bvrbg\""
Jul 17 10:57:33.654630 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.654605    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 10:57:33.747365 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.747324    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:33.747557 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:33.747416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:33.747557 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.747483    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 10:57:33.747557 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.747506    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7576d667b4-h5t84: secret "image-registry-tls" not found
Jul 17 10:57:33.747557 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.747541    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 10:57:33.747772 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.747585    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls podName:e973607b-b4cd-4b33-890f-b368d1e7e03a nodeName:}" failed. No retries permitted until 2026-07-17 10:57:34.747562965 +0000 UTC m=+37.604646908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls") pod "image-registry-7576d667b4-h5t84" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a") : secret "image-registry-tls" not found
Jul 17 10:57:33.747772 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:33.747616    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls podName:679aabef-721d-436b-977a-87f5d0608638 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:34.747597126 +0000 UTC m=+37.604681050 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls") pod "dns-default-7hgrl" (UID: "679aabef-721d-436b-977a-87f5d0608638") : secret "dns-default-metrics-tls" not found
Jul 17 10:57:34.521962 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:34.521943    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-47rdw"]
Jul 17 10:57:34.524761 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:57:34.524734    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod289bbdfa_ebd8_42af_be5d_95b7702a39f4.slice/crio-96df78b483c90e021f5846da6267975ed23ec4caeeae1953966d544ea710a67b WatchSource:0}: Error finding container 96df78b483c90e021f5846da6267975ed23ec4caeeae1953966d544ea710a67b: Status 404 returned error can't find the container with id 96df78b483c90e021f5846da6267975ed23ec4caeeae1953966d544ea710a67b
Jul 17 10:57:34.756695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:34.756651    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:34.756892 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:34.756759    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:34.756892 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:34.756825    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 10:57:34.756892 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:34.756871    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 10:57:34.756892 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:34.756884    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7576d667b4-h5t84: secret "image-registry-tls" not found
Jul 17 10:57:34.757117 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:34.756896    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls podName:679aabef-721d-436b-977a-87f5d0608638 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:36.756879603 +0000 UTC m=+39.613963543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls") pod "dns-default-7hgrl" (UID: "679aabef-721d-436b-977a-87f5d0608638") : secret "dns-default-metrics-tls" not found
Jul 17 10:57:34.757117 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:34.756932    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls podName:e973607b-b4cd-4b33-890f-b368d1e7e03a nodeName:}" failed. No retries permitted until 2026-07-17 10:57:36.75691756 +0000 UTC m=+39.614001504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls") pod "image-registry-7576d667b4-h5t84" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a") : secret "image-registry-tls" not found
Jul 17 10:57:34.820464 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:34.820432    2566 generic.go:356] "Generic (PLEG): container finished" podID="403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561" containerID="29da5be71cb8f81b080f689b9b392eb0791743203e4768f2ffc6224ca32abfe7" exitCode=0
Jul 17 10:57:34.820614 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:34.820509    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-skm5h" event={"ID":"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561","Type":"ContainerDied","Data":"29da5be71cb8f81b080f689b9b392eb0791743203e4768f2ffc6224ca32abfe7"}
Jul 17 10:57:34.821753 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:34.821729    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-47rdw" event={"ID":"289bbdfa-ebd8-42af-be5d-95b7702a39f4","Type":"ContainerStarted","Data":"96df78b483c90e021f5846da6267975ed23ec4caeeae1953966d544ea710a67b"}
Jul 17 10:57:35.829051 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:35.829010    2566 generic.go:356] "Generic (PLEG): container finished" podID="403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561" containerID="865ec29745b85374ca2f6255ca47e8e47d2682587f407539a681675d48ca0e5a" exitCode=0
Jul 17 10:57:35.829520 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:35.829088    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-skm5h" event={"ID":"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561","Type":"ContainerDied","Data":"865ec29745b85374ca2f6255ca47e8e47d2682587f407539a681675d48ca0e5a"}
Jul 17 10:57:36.033829 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:36.033795    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log" limit=1024
Jul 17 10:57:36.147335 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:36.147203    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log" limit=1024
Jul 17 10:57:36.230864 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:36.230828    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log" limit=1024
Jul 17 10:57:36.773777 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:36.773747    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:36.773947 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:36.773825    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:36.773947 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:36.773903    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 10:57:36.774057 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:36.773974    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls podName:679aabef-721d-436b-977a-87f5d0608638 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:40.773957133 +0000 UTC m=+43.631041069 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls") pod "dns-default-7hgrl" (UID: "679aabef-721d-436b-977a-87f5d0608638") : secret "dns-default-metrics-tls" not found
Jul 17 10:57:36.774057 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:36.773989    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 10:57:36.774057 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:36.774011    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7576d667b4-h5t84: secret "image-registry-tls" not found
Jul 17 10:57:36.774180 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:36.774092    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls podName:e973607b-b4cd-4b33-890f-b368d1e7e03a nodeName:}" failed. No retries permitted until 2026-07-17 10:57:40.774072138 +0000 UTC m=+43.631156082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls") pod "image-registry-7576d667b4-h5t84" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a") : secret "image-registry-tls" not found
Jul 17 10:57:36.834248 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:36.834221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-skm5h" event={"ID":"403c6ed9-8bd8-4b0a-bce9-e4d2ea66d561","Type":"ContainerStarted","Data":"a7b44bbbce86fc54b878f63fea5b4572e12726c2eceb9af9a1a8b9f38ce0224e"}
Jul 17 10:57:36.859383 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:36.859344    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-skm5h" podStartSLOduration=6.766782151 podStartE2EDuration="38.859328704s" totalImagesPullingTime="32.092546553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:36.857532767 +0000 UTC m=+39.714616712" watchObservedRunningTime="2026-07-17 10:57:36.859328704 +0000 UTC m=+39.716412648"
Jul 17 10:57:36.982714 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:36.982682    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log" limit=1024
Jul 17 10:57:37.051559 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:37.051471    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log" limit=1024
Jul 17 10:57:37.367790 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:37.367756    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log" limit=1024
Jul 17 10:57:37.449460 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:37.449415    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log" limit=1024
Jul 17 10:57:37.838346 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:37.838316    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-47rdw" event={"ID":"289bbdfa-ebd8-42af-be5d-95b7702a39f4","Type":"ContainerStarted","Data":"b7e7ccbd8beca4c2ba00e7a4f967bcc079ee84f66d5aac290ee499273a64d07c"}
Jul 17 10:57:37.866246 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:37.866196    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-47rdw" podStartSLOduration=3.555409649 podStartE2EDuration="5.866181325s" totalImagesPullingTime="2.310771676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:37.865699032 +0000 UTC m=+40.722782988" watchObservedRunningTime="2026-07-17 10:57:37.866181325 +0000 UTC m=+40.723265311"
Jul 17 10:57:40.801760 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:40.801725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:40.802111 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:40.801784    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:40.802111 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:40.801878    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 10:57:40.802111 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:40.801947    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls podName:679aabef-721d-436b-977a-87f5d0608638 nodeName:}" failed. No retries permitted until 2026-07-17 10:57:48.801932182 +0000 UTC m=+51.659016104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls") pod "dns-default-7hgrl" (UID: "679aabef-721d-436b-977a-87f5d0608638") : secret "dns-default-metrics-tls" not found
Jul 17 10:57:40.802111 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:40.801881    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 10:57:40.802111 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:40.801986    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7576d667b4-h5t84: secret "image-registry-tls" not found
Jul 17 10:57:40.802111 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:40.802027    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls podName:e973607b-b4cd-4b33-890f-b368d1e7e03a nodeName:}" failed. No retries permitted until 2026-07-17 10:57:48.802015996 +0000 UTC m=+51.659099934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls") pod "image-registry-7576d667b4-h5t84" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a") : secret "image-registry-tls" not found
Jul 17 10:57:48.237771 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:48.237735    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log" limit=1024
Jul 17 10:57:48.344631 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:48.344588    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log" limit=1024
Jul 17 10:57:48.855464 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:48.855432    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:57:48.855624 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:48.855489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:57:48.855624 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:48.855573    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 10:57:48.855624 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:48.855595    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 10:57:48.855624 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:48.855606    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7576d667b4-h5t84: secret "image-registry-tls" not found
Jul 17 10:57:48.855775 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:48.855637    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls podName:679aabef-721d-436b-977a-87f5d0608638 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:04.855621018 +0000 UTC m=+67.712704941 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls") pod "dns-default-7hgrl" (UID: "679aabef-721d-436b-977a-87f5d0608638") : secret "dns-default-metrics-tls" not found
Jul 17 10:57:48.855775 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:57:48.855651    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls podName:e973607b-b4cd-4b33-890f-b368d1e7e03a nodeName:}" failed. No retries permitted until 2026-07-17 10:58:04.85564471 +0000 UTC m=+67.712728633 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls") pod "image-registry-7576d667b4-h5t84" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a") : secret "image-registry-tls" not found
Jul 17 10:57:58.817190 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:57:58.817158    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sl2kk"
Jul 17 10:58:03.455780 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.455732    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:58:03.456194 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.455788    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:58:03.456194 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.455817    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:58:03.458441 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.458418    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 10:58:03.458588 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.458570    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 10:58:03.458669 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.458652    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 10:58:03.466938 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:03.466910    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 10:58:03.467056 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:03.466993    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs podName:4b9c0e03-0542-4a98-901e-fddc777badf8 nodeName:}" failed. No retries permitted until 2026-07-17 10:59:07.466972823 +0000 UTC m=+130.324056746 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs") pod "network-metrics-daemon-zpgj8" (UID: "4b9c0e03-0542-4a98-901e-fddc777badf8") : secret "metrics-daemon-secret" not found
Jul 17 10:58:03.469069 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.469053    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 10:58:03.470206 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.470188    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0758dbfc-ac21-4268-83e7-1cb5053415b8-original-pull-secret\") pod \"global-pull-secret-syncer-8kq7q\" (UID: \"0758dbfc-ac21-4268-83e7-1cb5053415b8\") " pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:58:03.479502 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.479484    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnw2h\" (UniqueName: \"kubernetes.io/projected/acd844fd-aed6-46c2-bd12-43fd4e7298fd-kube-api-access-xnw2h\") pod \"network-check-target-m86vn\" (UID: \"acd844fd-aed6-46c2-bd12-43fd4e7298fd\") " pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:58:03.665560 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.665533    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bvrbg\""
Jul 17 10:58:03.673024 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.672996    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:58:03.675306 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.675286    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kq7q"
Jul 17 10:58:03.800465 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.800432    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8kq7q"]
Jul 17 10:58:03.804056 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:58:03.804031    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0758dbfc_ac21_4268_83e7_1cb5053415b8.slice/crio-57594ced0f616951c8db0c6a684d593566e59938b3991e243d243066e1008bb5 WatchSource:0}: Error finding container 57594ced0f616951c8db0c6a684d593566e59938b3991e243d243066e1008bb5: Status 404 returned error can't find the container with id 57594ced0f616951c8db0c6a684d593566e59938b3991e243d243066e1008bb5
Jul 17 10:58:03.816916 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.816895    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m86vn"]
Jul 17 10:58:03.819684 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:58:03.819662    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd844fd_aed6_46c2_bd12_43fd4e7298fd.slice/crio-c0185b31084f31cb182f5401df635296fa0809508b83dde00b8775a86b56de16 WatchSource:0}: Error finding container c0185b31084f31cb182f5401df635296fa0809508b83dde00b8775a86b56de16: Status 404 returned error can't find the container with id c0185b31084f31cb182f5401df635296fa0809508b83dde00b8775a86b56de16
Jul 17 10:58:03.894795 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.894767    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8kq7q" event={"ID":"0758dbfc-ac21-4268-83e7-1cb5053415b8","Type":"ContainerStarted","Data":"57594ced0f616951c8db0c6a684d593566e59938b3991e243d243066e1008bb5"}
Jul 17 10:58:03.895724 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:03.895672    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m86vn" event={"ID":"acd844fd-aed6-46c2-bd12-43fd4e7298fd","Type":"ContainerStarted","Data":"c0185b31084f31cb182f5401df635296fa0809508b83dde00b8775a86b56de16"}
Jul 17 10:58:04.865461 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:04.865425    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:58:04.865951 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:04.865499    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:58:04.865951 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:04.865597    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 10:58:04.865951 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:04.865615    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 10:58:04.865951 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:04.865623    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7576d667b4-h5t84: secret "image-registry-tls" not found
Jul 17 10:58:04.865951 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:04.865683    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls podName:679aabef-721d-436b-977a-87f5d0608638 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:36.865665204 +0000 UTC m=+99.722749141 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls") pod "dns-default-7hgrl" (UID: "679aabef-721d-436b-977a-87f5d0608638") : secret "dns-default-metrics-tls" not found
Jul 17 10:58:04.865951 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:04.865720    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls podName:e973607b-b4cd-4b33-890f-b368d1e7e03a nodeName:}" failed. No retries permitted until 2026-07-17 10:58:36.865692992 +0000 UTC m=+99.722776917 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls") pod "image-registry-7576d667b4-h5t84" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a") : secret "image-registry-tls" not found
Jul 17 10:58:08.911432 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:08.911325    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8kq7q" event={"ID":"0758dbfc-ac21-4268-83e7-1cb5053415b8","Type":"ContainerStarted","Data":"12739d16df85bfc069b255f8632f83cb75eb3c078cc681151dab5e3273505933"}
Jul 17 10:58:08.912726 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:08.912683    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m86vn" event={"ID":"acd844fd-aed6-46c2-bd12-43fd4e7298fd","Type":"ContainerStarted","Data":"1e76d007e06be39b9960d838b81963c33900a69bae64d4d8f405c47ffab21725"}
Jul 17 10:58:08.912843 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:08.912815    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:58:08.928912 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:08.928873    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8kq7q" podStartSLOduration=66.143851845 podStartE2EDuration="1m10.928862326s" totalImagesPullingTime="4.785010481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:08.927328698 +0000 UTC m=+71.784412674" watchObservedRunningTime="2026-07-17 10:58:08.928862326 +0000 UTC m=+71.785946270"
Jul 17 10:58:08.943593 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:08.943555    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m86vn" podStartSLOduration=66.179027547 podStartE2EDuration="1m10.943545436s" totalImagesPullingTime="4.764517889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:08.943342924 +0000 UTC m=+71.800426888" watchObservedRunningTime="2026-07-17 10:58:08.943545436 +0000 UTC m=+71.800629371"
Jul 17 10:58:36.872918 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:36.872886    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:58:36.873417 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:36.872949    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") pod \"image-registry-7576d667b4-h5t84\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") " pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:58:36.873417 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:36.873048    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 10:58:36.873417 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:36.873118    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls podName:679aabef-721d-436b-977a-87f5d0608638 nodeName:}" failed. No retries permitted until 2026-07-17 10:59:40.873103076 +0000 UTC m=+163.730186999 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls") pod "dns-default-7hgrl" (UID: "679aabef-721d-436b-977a-87f5d0608638") : secret "dns-default-metrics-tls" not found
Jul 17 10:58:36.873417 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:36.873054    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 10:58:36.873417 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:36.873148    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7576d667b4-h5t84: secret "image-registry-tls" not found
Jul 17 10:58:36.873417 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:36.873201    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls podName:e973607b-b4cd-4b33-890f-b368d1e7e03a nodeName:}" failed. No retries permitted until 2026-07-17 10:59:40.873188758 +0000 UTC m=+163.730272680 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls") pod "image-registry-7576d667b4-h5t84" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a") : secret "image-registry-tls" not found
Jul 17 10:58:39.917927 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:39.917897    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m86vn"
Jul 17 10:58:42.865155 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.865121    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd"]
Jul 17 10:58:42.868339 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.868320    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd"
Jul 17 10:58:42.870841 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.870820    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vj8zz\""
Jul 17 10:58:42.877773 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.877754    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd"]
Jul 17 10:58:42.914180 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.914149    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsmp7\" (UniqueName: \"kubernetes.io/projected/7a45a411-6ac2-45a8-b24d-659d1e21e42f-kube-api-access-hsmp7\") pod \"network-check-source-7bc5b6bf85-5b4gd\" (UID: \"7a45a411-6ac2-45a8-b24d-659d1e21e42f\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd"
Jul 17 10:58:42.977535 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.977502    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-mf2tf"]
Jul 17 10:58:42.979298 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.979283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:42.982114 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.982093    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wsvx7\""
Jul 17 10:58:42.982233 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.982093    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 17 10:58:42.982354 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.982339    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 17 10:58:42.982446 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.982429    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 17 10:58:42.982507 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.982483    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 17 10:58:42.987989 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.987972    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 17 10:58:42.990599 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:42.990581    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-mf2tf"]
Jul 17 10:58:43.014617 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.014589    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsmp7\" (UniqueName: \"kubernetes.io/projected/7a45a411-6ac2-45a8-b24d-659d1e21e42f-kube-api-access-hsmp7\") pod \"network-check-source-7bc5b6bf85-5b4gd\" (UID: \"7a45a411-6ac2-45a8-b24d-659d1e21e42f\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd"
Jul 17 10:58:43.014792 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.014623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-trusted-ca\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.014792 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.014659    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cnj8\" (UniqueName: \"kubernetes.io/projected/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-kube-api-access-5cnj8\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.014792 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.014694    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-config\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.014792 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.014776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-serving-cert\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.035611 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.035588    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsmp7\" (UniqueName: \"kubernetes.io/projected/7a45a411-6ac2-45a8-b24d-659d1e21e42f-kube-api-access-hsmp7\") pod \"network-check-source-7bc5b6bf85-5b4gd\" (UID: \"7a45a411-6ac2-45a8-b24d-659d1e21e42f\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd"
Jul 17 10:58:43.115929 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.115830    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-trusted-ca\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.115929 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.115900    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cnj8\" (UniqueName: \"kubernetes.io/projected/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-kube-api-access-5cnj8\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.115929 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.115938    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-config\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.116174 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.116042    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-serving-cert\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.116641 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.116609    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-config\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.116777 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.116750    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-trusted-ca\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.118364 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.118335    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-serving-cert\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.127721 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.127689    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cnj8\" (UniqueName: \"kubernetes.io/projected/57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d-kube-api-access-5cnj8\") pod \"console-operator-565b7b569d-mf2tf\" (UID: \"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d\") " pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.176607 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.176572    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd"
Jul 17 10:58:43.289189 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.289159    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:43.292933 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.292911    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd"]
Jul 17 10:58:43.296014 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:58:43.295989    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a45a411_6ac2_45a8_b24d_659d1e21e42f.slice/crio-99265f283a67ac6e9703afd3517b09d16602336ce4e11743c4592262f38503d2 WatchSource:0}: Error finding container 99265f283a67ac6e9703afd3517b09d16602336ce4e11743c4592262f38503d2: Status 404 returned error can't find the container with id 99265f283a67ac6e9703afd3517b09d16602336ce4e11743c4592262f38503d2
Jul 17 10:58:43.321878 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.321850    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"]
Jul 17 10:58:43.324280 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.324221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:43.326739 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.326698    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 17 10:58:43.326836 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.326753    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rp5cn\""
Jul 17 10:58:43.326836 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.326774    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 17 10:58:43.335890 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.335296    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"]
Jul 17 10:58:43.414611 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.414580    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-mf2tf"]
Jul 17 10:58:43.417685 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:58:43.417663    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e7a55d_d9b5_4406_aa5c_7ce1b40ee65d.slice/crio-426c494513e6ee7236f7a46118155b707dccb79312fc08f1fbf81bc6fe55dee5 WatchSource:0}: Error finding container 426c494513e6ee7236f7a46118155b707dccb79312fc08f1fbf81bc6fe55dee5: Status 404 returned error can't find the container with id 426c494513e6ee7236f7a46118155b707dccb79312fc08f1fbf81bc6fe55dee5
Jul 17 10:58:43.418537 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.418515    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:43.418636 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.418552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:43.519158 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.519124    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:43.519158 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.519158    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:43.519385 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:43.519351    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 10:58:43.519441 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:43.519434    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert podName:e7d3da95-5ede-4a1f-bd2f-119ca091ecdd nodeName:}" failed. No retries permitted until 2026-07-17 10:58:44.019412036 +0000 UTC m=+106.876495970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-889sw" (UID: "e7d3da95-5ede-4a1f-bd2f-119ca091ecdd") : secret "networking-console-plugin-cert" not found
Jul 17 10:58:43.519797 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.519778    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:43.995273 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.995234    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd" event={"ID":"7a45a411-6ac2-45a8-b24d-659d1e21e42f","Type":"ContainerStarted","Data":"35851c9fee4d3a262b6cecf42ecb4a78ea058c8554c4f352c5eedae70492b269"}
Jul 17 10:58:43.995729 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.995295    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd" event={"ID":"7a45a411-6ac2-45a8-b24d-659d1e21e42f","Type":"ContainerStarted","Data":"99265f283a67ac6e9703afd3517b09d16602336ce4e11743c4592262f38503d2"}
Jul 17 10:58:43.996348 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:43.996321    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf" event={"ID":"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d","Type":"ContainerStarted","Data":"426c494513e6ee7236f7a46118155b707dccb79312fc08f1fbf81bc6fe55dee5"}
Jul 17 10:58:44.014292 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:44.014251    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5b4gd" podStartSLOduration=2.014237776 podStartE2EDuration="2.014237776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:44.013561419 +0000 UTC m=+106.870645366" watchObservedRunningTime="2026-07-17 10:58:44.014237776 +0000 UTC m=+106.871321721"
Jul 17 10:58:44.022861 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:44.022826    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:44.022965 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:44.022932    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 10:58:44.023004 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:44.022998    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert podName:e7d3da95-5ede-4a1f-bd2f-119ca091ecdd nodeName:}" failed. No retries permitted until 2026-07-17 10:58:45.022984251 +0000 UTC m=+107.880068174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-889sw" (UID: "e7d3da95-5ede-4a1f-bd2f-119ca091ecdd") : secret "networking-console-plugin-cert" not found
Jul 17 10:58:45.030532 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:45.030498    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:45.031019 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:45.030668    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 10:58:45.031019 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:45.030766    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert podName:e7d3da95-5ede-4a1f-bd2f-119ca091ecdd nodeName:}" failed. No retries permitted until 2026-07-17 10:58:47.030744124 +0000 UTC m=+109.887828047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-889sw" (UID: "e7d3da95-5ede-4a1f-bd2f-119ca091ecdd") : secret "networking-console-plugin-cert" not found
Jul 17 10:58:46.005865 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.005837    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/0.log"
Jul 17 10:58:46.006017 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.005876    2566 generic.go:356] "Generic (PLEG): container finished" podID="57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d" containerID="665d4c6d52436fb3a26089bba6246e5891eec47756209002e7d0d72e63f8f03a" exitCode=255
Jul 17 10:58:46.006017 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.005907    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf" event={"ID":"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d","Type":"ContainerDied","Data":"665d4c6d52436fb3a26089bba6246e5891eec47756209002e7d0d72e63f8f03a"}
Jul 17 10:58:46.006129 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.006114    2566 scope.go:117] "RemoveContainer" containerID="665d4c6d52436fb3a26089bba6246e5891eec47756209002e7d0d72e63f8f03a"
Jul 17 10:58:46.265569 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.265498    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d"]
Jul 17 10:58:46.267628 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.267612    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d"
Jul 17 10:58:46.270312 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.270279    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 17 10:58:46.270490 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.270455    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 17 10:58:46.270639 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.270610    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vx4l4\""
Jul 17 10:58:46.279120 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.279102    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d"]
Jul 17 10:58:46.341194 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.341165    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqww2\" (UniqueName: \"kubernetes.io/projected/d5fe56a1-f8b0-4583-ab74-817cb77a6a2f-kube-api-access-xqww2\") pod \"migrator-7cc9c7ffd7-hmd6d\" (UID: \"d5fe56a1-f8b0-4583-ab74-817cb77a6a2f\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d"
Jul 17 10:58:46.442599 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.442560    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqww2\" (UniqueName: \"kubernetes.io/projected/d5fe56a1-f8b0-4583-ab74-817cb77a6a2f-kube-api-access-xqww2\") pod \"migrator-7cc9c7ffd7-hmd6d\" (UID: \"d5fe56a1-f8b0-4583-ab74-817cb77a6a2f\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d"
Jul 17 10:58:46.452715 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.452678    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqww2\" (UniqueName: \"kubernetes.io/projected/d5fe56a1-f8b0-4583-ab74-817cb77a6a2f-kube-api-access-xqww2\") pod \"migrator-7cc9c7ffd7-hmd6d\" (UID: \"d5fe56a1-f8b0-4583-ab74-817cb77a6a2f\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d"
Jul 17 10:58:46.576485 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.576456    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d"
Jul 17 10:58:46.690868 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:46.690782    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d"]
Jul 17 10:58:46.693130 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:58:46.693100    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5fe56a1_f8b0_4583_ab74_817cb77a6a2f.slice/crio-9ec10d58ae381f38c999b64f2482354738b22bef8061df4983740f9ca2f17a5d WatchSource:0}: Error finding container 9ec10d58ae381f38c999b64f2482354738b22bef8061df4983740f9ca2f17a5d: Status 404 returned error can't find the container with id 9ec10d58ae381f38c999b64f2482354738b22bef8061df4983740f9ca2f17a5d
Jul 17 10:58:47.010061 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.009979    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d" event={"ID":"d5fe56a1-f8b0-4583-ab74-817cb77a6a2f","Type":"ContainerStarted","Data":"9ec10d58ae381f38c999b64f2482354738b22bef8061df4983740f9ca2f17a5d"}
Jul 17 10:58:47.011336 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.011320    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 10:58:47.011695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.011680    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/0.log"
Jul 17 10:58:47.011763 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.011728    2566 generic.go:356] "Generic (PLEG): container finished" podID="57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d" containerID="049f9470f34d3bb415af29daa908673b1e71d81275c8dbb6cdb3b2fadd51369c" exitCode=255
Jul 17 10:58:47.011799 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.011764    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf" event={"ID":"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d","Type":"ContainerDied","Data":"049f9470f34d3bb415af29daa908673b1e71d81275c8dbb6cdb3b2fadd51369c"}
Jul 17 10:58:47.011829 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.011812    2566 scope.go:117] "RemoveContainer" containerID="665d4c6d52436fb3a26089bba6246e5891eec47756209002e7d0d72e63f8f03a"
Jul 17 10:58:47.012003 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.011986    2566 scope.go:117] "RemoveContainer" containerID="049f9470f34d3bb415af29daa908673b1e71d81275c8dbb6cdb3b2fadd51369c"
Jul 17 10:58:47.012171 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.012147    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-mf2tf_openshift-console-operator(57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d)\"" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf" podUID="57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d"
Jul 17 10:58:47.047047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.047018    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:47.047146 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.047132    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 10:58:47.047217 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.047191    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert podName:e7d3da95-5ede-4a1f-bd2f-119ca091ecdd nodeName:}" failed. No retries permitted until 2026-07-17 10:58:51.0471718 +0000 UTC m=+113.904255725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-889sw" (UID: "e7d3da95-5ede-4a1f-bd2f-119ca091ecdd") : secret "networking-console-plugin-cert" not found
Jul 17 10:58:47.075617 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.075596    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rgk7n"]
Jul 17 10:58:47.078409 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.078394    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.080913 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.080888    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 10:58:47.081008 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.080917    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 10:58:47.081008 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.080935    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gfnlk\""
Jul 17 10:58:47.081008 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.080936    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 10:58:47.081181 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.081165    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 10:58:47.099397 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.099373    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rgk7n"]
Jul 17 10:58:47.148035 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.148006    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-crio-socket\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.148137 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.148040    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.148137 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.148122    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.148203 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.148163    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-data-volume\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.148203 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.148199    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-482tw\" (UniqueName: \"kubernetes.io/projected/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-api-access-482tw\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.248756 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.248700    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-data-volume\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.248935 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.248793    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-482tw\" (UniqueName: \"kubernetes.io/projected/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-api-access-482tw\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.248935 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.248841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-crio-socket\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.248935 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.248877    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.249080 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.248995    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 10:58:47.249080 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.249016    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.249163 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.249095    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:47.7490743 +0000 UTC m=+110.606158238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : configmap "kube-rbac-proxy" not found
Jul 17 10:58:47.249163 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.249114    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-data-volume\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.249163 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.248993    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-crio-socket\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.249163 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.249135    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:47.249333 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.249232    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:47.749211456 +0000 UTC m=+110.606295393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:47.259919 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.259896    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-482tw\" (UniqueName: \"kubernetes.io/projected/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-api-access-482tw\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.753613 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.753568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.753981 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:47.753663    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:47.753981 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.753722    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 10:58:47.753981 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.753797    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:48.753776804 +0000 UTC m=+111.610860746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : configmap "kube-rbac-proxy" not found
Jul 17 10:58:47.753981 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.753797    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:47.753981 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:47.753832    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:48.753822551 +0000 UTC m=+111.610906475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:48.016390 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:48.016319    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 10:58:48.016691 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:48.016662    2566 scope.go:117] "RemoveContainer" containerID="049f9470f34d3bb415af29daa908673b1e71d81275c8dbb6cdb3b2fadd51369c"
Jul 17 10:58:48.016853 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:48.016832    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-mf2tf_openshift-console-operator(57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d)\"" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf" podUID="57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d"
Jul 17 10:58:48.018131 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:48.018111    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d" event={"ID":"d5fe56a1-f8b0-4583-ab74-817cb77a6a2f","Type":"ContainerStarted","Data":"222d4b1b1a8dd3abe7f3dd77b7e512e5a1ad5e9b670f9f4ddaa0381b9bd9d4ae"}
Jul 17 10:58:48.018228 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:48.018155    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d" event={"ID":"d5fe56a1-f8b0-4583-ab74-817cb77a6a2f","Type":"ContainerStarted","Data":"6bcdda6bf08d2fd16da00c3edfffbb0659cf74148c8e0cc97a072b5395350dd3"}
Jul 17 10:58:48.074441 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:48.074395    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hmd6d" podStartSLOduration=0.994893302 podStartE2EDuration="2.074382187s" totalImagesPullingTime="1.079488885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:48.073491134 +0000 UTC m=+110.930575105" watchObservedRunningTime="2026-07-17 10:58:48.074382187 +0000 UTC m=+110.931466131"
Jul 17 10:58:48.759798 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:48.759764    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:48.760169 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:48.759825    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:48.760169 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:48.759877    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 10:58:48.760169 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:48.759930    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:50.759916963 +0000 UTC m=+113.617000890 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : configmap "kube-rbac-proxy" not found
Jul 17 10:58:48.760169 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:48.759975    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:48.760169 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:48.760025    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:50.760013268 +0000 UTC m=+113.617097191 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:50.094510 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:50.094483    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rxwbd_6aefeda7-ce38-4020-8d0b-6ea66c2d3d36/dns-node-resolver/0.log"
Jul 17 10:58:50.775543 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:50.775499    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:50.775764 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:50.775621    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 10:58:50.775764 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:50.775695    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:54.775672228 +0000 UTC m=+117.632756151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : configmap "kube-rbac-proxy" not found
Jul 17 10:58:50.775764 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:50.775754    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:50.775942 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:50.775879    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:50.775942 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:50.775928    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:54.775916361 +0000 UTC m=+117.633000283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:50.891682 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:50.891655    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pf6hp_cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e/node-ca/0.log"
Jul 17 10:58:51.078835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:51.078796    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:51.079004 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:51.078947    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 10:58:51.079051 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:51.079021    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert podName:e7d3da95-5ede-4a1f-bd2f-119ca091ecdd nodeName:}" failed. No retries permitted until 2026-07-17 10:58:59.07900556 +0000 UTC m=+121.936089483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-889sw" (UID: "e7d3da95-5ede-4a1f-bd2f-119ca091ecdd") : secret "networking-console-plugin-cert" not found
Jul 17 10:58:51.501956 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:51.501887    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-47rdw_289bbdfa-ebd8-42af-be5d-95b7702a39f4/serve-healthcheck-canary/0.log"
Jul 17 10:58:52.092911 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:52.092884    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-hmd6d_d5fe56a1-f8b0-4583-ab74-817cb77a6a2f/migrator/0.log"
Jul 17 10:58:52.292128 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:52.292095    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-hmd6d_d5fe56a1-f8b0-4583-ab74-817cb77a6a2f/graceful-termination/0.log"
Jul 17 10:58:53.290246 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:53.290212    2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:53.290654 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:53.290298    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:58:53.290700 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:53.290664    2566 scope.go:117] "RemoveContainer" containerID="049f9470f34d3bb415af29daa908673b1e71d81275c8dbb6cdb3b2fadd51369c"
Jul 17 10:58:53.290858 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:53.290838    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-mf2tf_openshift-console-operator(57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d)\"" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf" podUID="57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d"
Jul 17 10:58:54.806827 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:54.806776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:54.807225 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:54.806872    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:58:54.807225 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:54.806894    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 10:58:54.807225 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:54.806954    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:59:02.806938937 +0000 UTC m=+125.664022863 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : configmap "kube-rbac-proxy" not found
Jul 17 10:58:54.807225 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:54.807001    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:54.807225 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:54.807057    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls podName:ff832f52-dc1e-4629-9ba0-8f872f0ecbf4 nodeName:}" failed. No retries permitted until 2026-07-17 10:59:02.807045665 +0000 UTC m=+125.664129588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rgk7n" (UID: "ff832f52-dc1e-4629-9ba0-8f872f0ecbf4") : secret "insights-runtime-extractor-tls" not found
Jul 17 10:58:59.143841 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:58:59.143805    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:58:59.144230 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:59.143950    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 10:58:59.144230 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:58:59.144009    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert podName:e7d3da95-5ede-4a1f-bd2f-119ca091ecdd nodeName:}" failed. No retries permitted until 2026-07-17 10:59:15.143995307 +0000 UTC m=+138.001079230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-889sw" (UID: "e7d3da95-5ede-4a1f-bd2f-119ca091ecdd") : secret "networking-console-plugin-cert" not found
Jul 17 10:59:02.876912 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:02.876879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:59:02.877295 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:02.876940    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:59:02.877395 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:02.877374    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:59:02.879167 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:02.879150    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff832f52-dc1e-4629-9ba0-8f872f0ecbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rgk7n\" (UID: \"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4\") " pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:59:02.990338 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:02.990311    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gfnlk\""
Jul 17 10:59:02.997639 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:02.997620    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rgk7n"
Jul 17 10:59:03.121372 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:03.121347    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rgk7n"]
Jul 17 10:59:03.123603 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:59:03.123575    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff832f52_dc1e_4629_9ba0_8f872f0ecbf4.slice/crio-748546d7bd2b92947bb378255b4025130ca43e4ac73d3129618201f835bbaa7b WatchSource:0}: Error finding container 748546d7bd2b92947bb378255b4025130ca43e4ac73d3129618201f835bbaa7b: Status 404 returned error can't find the container with id 748546d7bd2b92947bb378255b4025130ca43e4ac73d3129618201f835bbaa7b
Jul 17 10:59:04.066847 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:04.066806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rgk7n" event={"ID":"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4","Type":"ContainerStarted","Data":"748546d7bd2b92947bb378255b4025130ca43e4ac73d3129618201f835bbaa7b"}
Jul 17 10:59:07.513926 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:07.513889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:59:07.516380 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:07.516354    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b9c0e03-0542-4a98-901e-fddc777badf8-metrics-certs\") pod \"network-metrics-daemon-zpgj8\" (UID: \"4b9c0e03-0542-4a98-901e-fddc777badf8\") " pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:59:07.573087 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:07.573062    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m4ccj\""
Jul 17 10:59:07.579837 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:07.579811    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zpgj8"
Jul 17 10:59:07.652981 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:07.652958    2566 scope.go:117] "RemoveContainer" containerID="049f9470f34d3bb415af29daa908673b1e71d81275c8dbb6cdb3b2fadd51369c"
Jul 17 10:59:07.914773 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:07.914749    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zpgj8"]
Jul 17 10:59:07.917598 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:59:07.917574    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b9c0e03_0542_4a98_901e_fddc777badf8.slice/crio-628282fabd2b7f422e70c277262bbc0c4646b52f3216e9fbcaf3e55d88f80864 WatchSource:0}: Error finding container 628282fabd2b7f422e70c277262bbc0c4646b52f3216e9fbcaf3e55d88f80864: Status 404 returned error can't find the container with id 628282fabd2b7f422e70c277262bbc0c4646b52f3216e9fbcaf3e55d88f80864
Jul 17 10:59:08.081136 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:08.081106    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rgk7n" event={"ID":"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4","Type":"ContainerStarted","Data":"4d8d03e296f13da16bda0bea8dceec14fa76b433ab56ecac6042f91edf7ab1d4"}
Jul 17 10:59:08.082239 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:08.082207    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zpgj8" event={"ID":"4b9c0e03-0542-4a98-901e-fddc777badf8","Type":"ContainerStarted","Data":"628282fabd2b7f422e70c277262bbc0c4646b52f3216e9fbcaf3e55d88f80864"}
Jul 17 10:59:08.083866 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:08.083848    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 10:59:08.083972 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:08.083920    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf" event={"ID":"57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d","Type":"ContainerStarted","Data":"a69aea535fdfc163ecaea0cb76d215380bcdd27036033ebf23ed96ea3709c505"}
Jul 17 10:59:08.084179 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:08.084156    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:59:08.109975 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:08.109926    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf" podStartSLOduration=24.122069951 podStartE2EDuration="26.109912444s" totalImagesPullingTime="1.987842493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:08.109130453 +0000 UTC m=+130.966214398" watchObservedRunningTime="2026-07-17 10:59:08.109912444 +0000 UTC m=+130.966996389"
Jul 17 10:59:08.611872 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:08.611838    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-mf2tf"
Jul 17 10:59:10.093752 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:10.093715    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zpgj8" event={"ID":"4b9c0e03-0542-4a98-901e-fddc777badf8","Type":"ContainerStarted","Data":"7cb3775d6d68bfda685e85f40c98cabd09f63f23361ade26469486a4a942ba93"}
Jul 17 10:59:10.094188 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:10.093784    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zpgj8" event={"ID":"4b9c0e03-0542-4a98-901e-fddc777badf8","Type":"ContainerStarted","Data":"ec101fb734f41c2bb1eb12d0a38d2e5d3e05237545416b791a25749868e7f6b3"}
Jul 17 10:59:10.119284 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:10.119234    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zpgj8" podStartSLOduration=130.798506232 podStartE2EDuration="2m12.119220285s" totalImagesPullingTime="1.320714053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:10.117797273 +0000 UTC m=+132.974881218" watchObservedRunningTime="2026-07-17 10:59:10.119220285 +0000 UTC m=+132.976304245"
Jul 17 10:59:12.102403 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:12.102366    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rgk7n" event={"ID":"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4","Type":"ContainerStarted","Data":"3f1c639e9fa927b0ef09b1e1fac51a49875b5d26c8036d7a4380af3db9b2d08d"}
Jul 17 10:59:15.113606 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:15.113569    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rgk7n" event={"ID":"ff832f52-dc1e-4629-9ba0-8f872f0ecbf4","Type":"ContainerStarted","Data":"4ffdee29e238f3c2ddd3eb293f9ee5de7224143b9512b85da49e36724be6edc9"}
Jul 17 10:59:15.133492 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:15.133383    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rgk7n" podStartSLOduration=16.63324237 podStartE2EDuration="28.133364675s" totalImagesPullingTime="11.500122305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:58:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:15.13312507 +0000 UTC m=+137.990209010" watchObservedRunningTime="2026-07-17 10:59:15.133364675 +0000 UTC m=+137.990448620"
Jul 17 10:59:15.181461 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:15.181432    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:59:15.183716 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:15.183675    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7d3da95-5ede-4a1f-bd2f-119ca091ecdd-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-889sw\" (UID: \"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:59:15.443775 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:15.443672    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rp5cn\""
Jul 17 10:59:15.451926 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:15.451898    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"
Jul 17 10:59:15.573840 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:15.573803    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-889sw"]
Jul 17 10:59:15.577516 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:59:15.577490    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d3da95_5ede_4a1f_bd2f_119ca091ecdd.slice/crio-002c04abc34c51c9def7e71cd4a246de651fad990e5c0cb04f28161df633c609 WatchSource:0}: Error finding container 002c04abc34c51c9def7e71cd4a246de651fad990e5c0cb04f28161df633c609: Status 404 returned error can't find the container with id 002c04abc34c51c9def7e71cd4a246de651fad990e5c0cb04f28161df633c609
Jul 17 10:59:16.117750 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:16.117701    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw" event={"ID":"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd","Type":"ContainerStarted","Data":"002c04abc34c51c9def7e71cd4a246de651fad990e5c0cb04f28161df633c609"}
Jul 17 10:59:17.122168 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:17.122138    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw" event={"ID":"e7d3da95-5ede-4a1f-bd2f-119ca091ecdd","Type":"ContainerStarted","Data":"2b62cded7b0e40529e8bfbea5bdd446586be031cae13c57be5560d76a01153c5"}
Jul 17 10:59:17.155689 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:17.155646    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-889sw" podStartSLOduration=32.949492309 podStartE2EDuration="34.155628569s" totalImagesPullingTime="1.20613626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:17.155489598 +0000 UTC m=+140.012573544" watchObservedRunningTime="2026-07-17 10:59:17.155628569 +0000 UTC m=+140.012712513"
Jul 17 10:59:28.117190 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.117158    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b4d588f69-m4vdd"]
Jul 17 10:59:28.121640 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.121620    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.124727 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.124692    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 10:59:28.124838 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.124814    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 10:59:28.125904 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.125884    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 10:59:28.126004 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.125931    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tnv9t\""
Jul 17 10:59:28.126004 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.125961    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 10:59:28.126120 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.125891    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 10:59:28.126168 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.126153    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 10:59:28.126247 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.126224    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 10:59:28.130974 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.130952    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 10:59:28.138629 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.138608    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4d588f69-m4vdd"]
Jul 17 10:59:28.284425 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.284388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-serving-cert\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.284597 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.284435    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-trusted-ca-bundle\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.284597 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.284463    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzz67\" (UniqueName: \"kubernetes.io/projected/5e74e109-9ff6-449b-bb64-9ab2336c450c-kube-api-access-xzz67\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.284597 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.284518    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-config\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.284597 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.284571    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-service-ca\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.284811 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.284617    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-oauth-config\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.284811 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.284659    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-oauth-serving-cert\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.385243 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.385175    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-serving-cert\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.385243 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.385211    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-trusted-ca-bundle\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.385243 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.385228    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzz67\" (UniqueName: \"kubernetes.io/projected/5e74e109-9ff6-449b-bb64-9ab2336c450c-kube-api-access-xzz67\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.385497 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.385249    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-config\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.385497 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.385290    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-service-ca\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.385497 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.385345    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-oauth-config\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.385497 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.385379    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-oauth-serving-cert\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.386143 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.386104    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-config\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.386265 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.386166    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-oauth-serving-cert\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.386265 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.386171    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-trusted-ca-bundle\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.386265 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.386196    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-service-ca\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.387741 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.387701    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-serving-cert\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.387835 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.387750    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-oauth-config\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.394950 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.394931    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzz67\" (UniqueName: \"kubernetes.io/projected/5e74e109-9ff6-449b-bb64-9ab2336c450c-kube-api-access-xzz67\") pod \"console-5b4d588f69-m4vdd\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") " pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.432735 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.432701    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:28.557210 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:28.557181    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4d588f69-m4vdd"]
Jul 17 10:59:28.559907 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:59:28.559881    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e74e109_9ff6_449b_bb64_9ab2336c450c.slice/crio-94a74c07ae16b73d63d5069e02d28394fa08e463601e07ae906cb36db3b01c18 WatchSource:0}: Error finding container 94a74c07ae16b73d63d5069e02d28394fa08e463601e07ae906cb36db3b01c18: Status 404 returned error can't find the container with id 94a74c07ae16b73d63d5069e02d28394fa08e463601e07ae906cb36db3b01c18
Jul 17 10:59:29.162404 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:29.162359    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4d588f69-m4vdd" event={"ID":"5e74e109-9ff6-449b-bb64-9ab2336c450c","Type":"ContainerStarted","Data":"94a74c07ae16b73d63d5069e02d28394fa08e463601e07ae906cb36db3b01c18"}
Jul 17 10:59:32.175014 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:32.174974    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4d588f69-m4vdd" event={"ID":"5e74e109-9ff6-449b-bb64-9ab2336c450c","Type":"ContainerStarted","Data":"3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f"}
Jul 17 10:59:32.194989 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:32.194945    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b4d588f69-m4vdd" podStartSLOduration=1.546808166 podStartE2EDuration="4.194930365s" totalImagesPullingTime="2.648122199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:32.194806536 +0000 UTC m=+155.051890481" watchObservedRunningTime="2026-07-17 10:59:32.194930365 +0000 UTC m=+155.052014312"
Jul 17 10:59:33.235340 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:33.235313    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7576d667b4-h5t84"]
Jul 17 10:59:33.235730 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:59:33.235476    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7576d667b4-h5t84" podUID="e973607b-b4cd-4b33-890f-b368d1e7e03a"
Jul 17 10:59:34.182176 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.182139    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:59:34.188850 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.188822    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:59:34.199330 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.199309    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8jmst"]
Jul 17 10:59:34.202240 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.202224    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.204807 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.204789    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 10:59:34.205008 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.204993    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 10:59:34.205125 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.205101    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tpqvh\""
Jul 17 10:59:34.205189 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.205175    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 10:59:34.205949 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.205932    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 10:59:34.205949 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.205947    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 10:59:34.206102 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.205947    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 10:59:34.233144 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233119    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-image-registry-private-configuration\") pod \"e973607b-b4cd-4b33-890f-b368d1e7e03a\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") "
Jul 17 10:59:34.233246 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233152    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-bound-sa-token\") pod \"e973607b-b4cd-4b33-890f-b368d1e7e03a\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") "
Jul 17 10:59:34.233246 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233187    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e973607b-b4cd-4b33-890f-b368d1e7e03a-ca-trust-extracted\") pod \"e973607b-b4cd-4b33-890f-b368d1e7e03a\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") "
Jul 17 10:59:34.233246 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233205    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-trusted-ca\") pod \"e973607b-b4cd-4b33-890f-b368d1e7e03a\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") "
Jul 17 10:59:34.233246 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233236    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qrfp\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-kube-api-access-2qrfp\") pod \"e973607b-b4cd-4b33-890f-b368d1e7e03a\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") "
Jul 17 10:59:34.233441 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233252    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-installation-pull-secrets\") pod \"e973607b-b4cd-4b33-890f-b368d1e7e03a\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") "
Jul 17 10:59:34.233441 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233292    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-certificates\") pod \"e973607b-b4cd-4b33-890f-b368d1e7e03a\" (UID: \"e973607b-b4cd-4b33-890f-b368d1e7e03a\") "
Jul 17 10:59:34.233441 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233385    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.233441 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233419    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzjl6\" (UniqueName: \"kubernetes.io/projected/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-kube-api-access-nzjl6\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.233441 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233430    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e973607b-b4cd-4b33-890f-b368d1e7e03a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e973607b-b4cd-4b33-890f-b368d1e7e03a" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:59:34.233695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233452    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-textfile\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.233695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233502    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-wtmp\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.233695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-sys\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.233695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-root\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.234261 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.234196    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e973607b-b4cd-4b33-890f-b368d1e7e03a" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:59:34.236207 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.233598    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-metrics-client-ca\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.236207 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.234461    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-tls\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.236207 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.234482    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e973607b-b4cd-4b33-890f-b368d1e7e03a" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:59:34.236207 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.234575    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.236207 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.234694    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e973607b-b4cd-4b33-890f-b368d1e7e03a-ca-trust-extracted\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 10:59:34.236207 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.234758    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-trusted-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 10:59:34.236207 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.234775    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-certificates\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 10:59:34.240577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.239850    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e973607b-b4cd-4b33-890f-b368d1e7e03a" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:59:34.240577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.239925    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e973607b-b4cd-4b33-890f-b368d1e7e03a" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:59:34.240577 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.240405    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-kube-api-access-2qrfp" (OuterVolumeSpecName: "kube-api-access-2qrfp") pod "e973607b-b4cd-4b33-890f-b368d1e7e03a" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a"). InnerVolumeSpecName "kube-api-access-2qrfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:59:34.240779 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.240633    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e973607b-b4cd-4b33-890f-b368d1e7e03a" (UID: "e973607b-b4cd-4b33-890f-b368d1e7e03a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:59:34.335742 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.335685    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.335917 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.335753    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.335917 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.335777    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzjl6\" (UniqueName: \"kubernetes.io/projected/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-kube-api-access-nzjl6\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336041 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.335910    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-textfile\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336041 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.335954    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-wtmp\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336041 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.335983    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-sys\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336041 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336017    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-root\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336229 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-metrics-client-ca\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336229 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336110    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-tls\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336229 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336110    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-root\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336229 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336135    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-sys\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336229 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336166    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-wtmp\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336229 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336186    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-image-registry-private-configuration\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 10:59:34.336229 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336214    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-bound-sa-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 10:59:34.336548 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336235    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qrfp\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-kube-api-access-2qrfp\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 10:59:34.336548 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336251    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e973607b-b4cd-4b33-890f-b368d1e7e03a-installation-pull-secrets\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 10:59:34.336548 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:59:34.336259    2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 17 10:59:34.336548 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336290    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-textfile\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336548 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:59:34.336318    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-tls podName:dfb33b7f-1020-45f8-b3a2-b7b23102e2d9 nodeName:}" failed. No retries permitted until 2026-07-17 10:59:34.836300346 +0000 UTC m=+157.693384286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-tls") pod "node-exporter-8jmst" (UID: "dfb33b7f-1020-45f8-b3a2-b7b23102e2d9") : secret "node-exporter-tls" not found
Jul 17 10:59:34.336548 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336445    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.336858 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.336744    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-metrics-client-ca\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.338046 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.338027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.347279 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.347253    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzjl6\" (UniqueName: \"kubernetes.io/projected/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-kube-api-access-nzjl6\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.840495 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.840462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-tls\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:34.842674 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:34.842655    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfb33b7f-1020-45f8-b3a2-b7b23102e2d9-node-exporter-tls\") pod \"node-exporter-8jmst\" (UID: \"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9\") " pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:35.111780 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.111680    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8jmst"
Jul 17 10:59:35.122989 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:59:35.122964    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb33b7f_1020_45f8_b3a2_b7b23102e2d9.slice/crio-f5388a229d20ae2f82ee7473a11de73d74041d6c64d70096ee4be852ebe2fc0d WatchSource:0}: Error finding container f5388a229d20ae2f82ee7473a11de73d74041d6c64d70096ee4be852ebe2fc0d: Status 404 returned error can't find the container with id f5388a229d20ae2f82ee7473a11de73d74041d6c64d70096ee4be852ebe2fc0d
Jul 17 10:59:35.185773 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.185745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jmst" event={"ID":"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9","Type":"ContainerStarted","Data":"f5388a229d20ae2f82ee7473a11de73d74041d6c64d70096ee4be852ebe2fc0d"}
Jul 17 10:59:35.185773 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.185771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7576d667b4-h5t84"
Jul 17 10:59:35.222610 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.222584    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7576d667b4-h5t84"]
Jul 17 10:59:35.225362 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.225337    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7576d667b4-h5t84"]
Jul 17 10:59:35.244047 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.244021    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e973607b-b4cd-4b33-890f-b368d1e7e03a-registry-tls\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 10:59:35.303530 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.303487    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 10:59:35.307931 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.307913    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.310828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.310806    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 17 10:59:35.310828 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.310822    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 17 10:59:35.310975 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.310806    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 17 10:59:35.311132 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.311113    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 17 10:59:35.311189 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.311168    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 17 10:59:35.311189 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.311176    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 17 10:59:35.311367 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.311349    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 17 10:59:35.311466 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.311364    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wfjxm\""
Jul 17 10:59:35.311466 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.311422    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 17 10:59:35.311641 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.311626    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 17 10:59:35.327928 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.327905    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 10:59:35.344328 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344304    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-web-config\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344412 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344330    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344412 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344356    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344412 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344412 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344400    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344573 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344573 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344462    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344573 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344487    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-out\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344573 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344689 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344585    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344689 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344614    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344689 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344645    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.344689 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.344665    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9lt5\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-kube-api-access-r9lt5\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445220 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445154    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445220 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445220 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445200    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9lt5\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-kube-api-access-r9lt5\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445425 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445236    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-web-config\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445425 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445257    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445531 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445435    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445531 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445477    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445531 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445680 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445541    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445680 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445680 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445612    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-out\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445680 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445647    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445912 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445691    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445912 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.445763    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.445912 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:59:35.445879    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-trusted-ca-bundle podName:6fd75c4e-9cc4-482e-ae32-04da804666bb nodeName:}" failed. No retries permitted until 2026-07-17 10:59:35.945858255 +0000 UTC m=+158.802942194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb") : configmap references non-existent config key: ca-bundle.crt
Jul 17 10:59:35.447423 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.447356    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.448461 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.448432    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.448557 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.448482    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.449353 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.449179    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-web-config\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.449436 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.449407    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-out\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.449566 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.449548    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.449693 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.449668    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.449693 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.449685    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.449996 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.449978    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.450188 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.450165    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.454390 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.454364    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9lt5\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-kube-api-access-r9lt5\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.655286 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.655256    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e973607b-b4cd-4b33-890f-b368d1e7e03a" path="/var/lib/kubelet/pods/e973607b-b4cd-4b33-890f-b368d1e7e03a/volumes"
Jul 17 10:59:35.950588 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.950547    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:35.951393 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:35.951370    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:36.027850 ip-10-0-134-138 kubenswrapper[2566]: E0717 10:59:36.027801    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7hgrl" podUID="679aabef-721d-436b-977a-87f5d0608638"
Jul 17 10:59:36.189147 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:36.189117    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:59:36.216978 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:36.216959    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 10:59:36.368611 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:36.368586    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 10:59:36.411434 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:59:36.411407    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd75c4e_9cc4_482e_ae32_04da804666bb.slice/crio-f948b24eee581156c44331ff83379dfb6b5a700c5c947277f0b7ba09c33c54dd WatchSource:0}: Error finding container f948b24eee581156c44331ff83379dfb6b5a700c5c947277f0b7ba09c33c54dd: Status 404 returned error can't find the container with id f948b24eee581156c44331ff83379dfb6b5a700c5c947277f0b7ba09c33c54dd
Jul 17 10:59:37.194412 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:37.194340    2566 generic.go:356] "Generic (PLEG): container finished" podID="dfb33b7f-1020-45f8-b3a2-b7b23102e2d9" containerID="e6094eb8e4c30897754766c85af9ccf5fc69993eb408b354429687d043532670" exitCode=0
Jul 17 10:59:37.194583 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:37.194443    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jmst" event={"ID":"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9","Type":"ContainerDied","Data":"e6094eb8e4c30897754766c85af9ccf5fc69993eb408b354429687d043532670"}
Jul 17 10:59:37.195773 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:37.195751    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerStarted","Data":"f948b24eee581156c44331ff83379dfb6b5a700c5c947277f0b7ba09c33c54dd"}
Jul 17 10:59:38.200889 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.200852    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jmst" event={"ID":"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9","Type":"ContainerStarted","Data":"5a659b7a4dd611fd4293b29554543481a29a56095dfdfa0c1d1d7504734326c3"}
Jul 17 10:59:38.201318 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.200915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jmst" event={"ID":"dfb33b7f-1020-45f8-b3a2-b7b23102e2d9","Type":"ContainerStarted","Data":"5f06457fbde2651b597e44936b7c346047fa08d38a0d70641cc90ff60bb3e972"}
Jul 17 10:59:38.202339 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.202316    2566 generic.go:356] "Generic (PLEG): container finished" podID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerID="f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2" exitCode=0
Jul 17 10:59:38.202441 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.202359    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerDied","Data":"f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2"}
Jul 17 10:59:38.225848 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.225803    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8jmst" podStartSLOduration=3.068137213 podStartE2EDuration="4.225789692s" totalImagesPullingTime="1.090378229s" totalInitContainerRuntime="67.27425ms" isStatefulPod=true podCreationTimestamp="2026-07-17 10:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:38.224801878 +0000 UTC m=+161.081885824" watchObservedRunningTime="2026-07-17 10:59:38.225789692 +0000 UTC m=+161.082873637"
Jul 17 10:59:38.433858 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.433820    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:38.434052 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.433877    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:38.438592 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.438570    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:38.987291 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.987258    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"]
Jul 17 10:59:38.990456 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.990429    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:38.993849 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.993815    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-t952l\""
Jul 17 10:59:38.994805 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.994784    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 17 10:59:38.999310 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:38.999287    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"]
Jul 17 10:59:39.076188 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.076157    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxw7p\" (UniqueName: \"kubernetes.io/projected/a9d66ff0-9de6-46f8-8742-5e66bf07d950-kube-api-access-sxw7p\") pod \"monitoring-plugin-7b7b7bc8bf-5fkfh\" (UID: \"a9d66ff0-9de6-46f8-8742-5e66bf07d950\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:39.076336 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.076314    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9d66ff0-9de6-46f8-8742-5e66bf07d950-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5fkfh\" (UID: \"a9d66ff0-9de6-46f8-8742-5e66bf07d950\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:39.177102 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.177068    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxw7p\" (UniqueName: \"kubernetes.io/projected/a9d66ff0-9de6-46f8-8742-5e66bf07d950-kube-api-access-sxw7p\") pod \"monitoring-plugin-7b7b7bc8bf-5fkfh\" (UID: \"a9d66ff0-9de6-46f8-8742-5e66bf07d950\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:39.177282 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.177255    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9d66ff0-9de6-46f8-8742-5e66bf07d950-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5fkfh\" (UID: \"a9d66ff0-9de6-46f8-8742-5e66bf07d950\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:39.180081 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.180058    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9d66ff0-9de6-46f8-8742-5e66bf07d950-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5fkfh\" (UID: \"a9d66ff0-9de6-46f8-8742-5e66bf07d950\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:39.186617 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.186594    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxw7p\" (UniqueName: \"kubernetes.io/projected/a9d66ff0-9de6-46f8-8742-5e66bf07d950-kube-api-access-sxw7p\") pod \"monitoring-plugin-7b7b7bc8bf-5fkfh\" (UID: \"a9d66ff0-9de6-46f8-8742-5e66bf07d950\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:39.215428 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.215404    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 10:59:39.300900 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.300837    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:39.422362 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.422334    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-55dffc6f4d-gd845"]
Jul 17 10:59:39.425740 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.425698    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.428662 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.428641    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-67g5t\""
Jul 17 10:59:39.428937 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.428917    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 17 10:59:39.429307 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.429291    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 17 10:59:39.429958 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.429942    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 17 10:59:39.430184 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.430169    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 17 10:59:39.430372 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.430359    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 17 10:59:39.437262 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.437243    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 17 10:59:39.442508 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.442490    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"]
Jul 17 10:59:39.446235 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.446216    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55dffc6f4d-gd845"]
Jul 17 10:59:39.479990 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.479964    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.480107 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.480015    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-telemeter-client-tls\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.480107 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.480053    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0367d82f-a576-4753-bf69-d536f4a8cfad-metrics-client-ca\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.480210 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.480152    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0367d82f-a576-4753-bf69-d536f4a8cfad-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.480262 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.480207    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-federate-client-tls\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.480262 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.480242    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0367d82f-a576-4753-bf69-d536f4a8cfad-serving-certs-ca-bundle\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.480331 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.480321    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-secret-telemeter-client\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.480414 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.480398    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmm2\" (UniqueName: \"kubernetes.io/projected/0367d82f-a576-4753-bf69-d536f4a8cfad-kube-api-access-5pmm2\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.581503 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.581471    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-secret-telemeter-client\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.581695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.581551    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmm2\" (UniqueName: \"kubernetes.io/projected/0367d82f-a576-4753-bf69-d536f4a8cfad-kube-api-access-5pmm2\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.581695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.581587    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.581695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.581625    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-telemeter-client-tls\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.581695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.581647    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0367d82f-a576-4753-bf69-d536f4a8cfad-metrics-client-ca\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.581695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.581672    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0367d82f-a576-4753-bf69-d536f4a8cfad-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.581695 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.581694    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-federate-client-tls\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.582053 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.581744    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0367d82f-a576-4753-bf69-d536f4a8cfad-serving-certs-ca-bundle\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.582531 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.582471    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0367d82f-a576-4753-bf69-d536f4a8cfad-metrics-client-ca\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.582826 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.582672    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0367d82f-a576-4753-bf69-d536f4a8cfad-serving-certs-ca-bundle\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.583245 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.583201    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0367d82f-a576-4753-bf69-d536f4a8cfad-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.584682 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.584655    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.584783 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.584664    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-secret-telemeter-client\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.584783 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.584745    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-telemeter-client-tls\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.585248 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.585221    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0367d82f-a576-4753-bf69-d536f4a8cfad-federate-client-tls\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.591918 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.591893    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmm2\" (UniqueName: \"kubernetes.io/projected/0367d82f-a576-4753-bf69-d536f4a8cfad-kube-api-access-5pmm2\") pod \"telemeter-client-55dffc6f4d-gd845\" (UID: \"0367d82f-a576-4753-bf69-d536f4a8cfad\") " pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.735895 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:59:39.735864    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d66ff0_9de6_46f8_8742_5e66bf07d950.slice/crio-1864c33b14f133a9024aa6a23c9aa00ba647cb3cc678448748451568c61ddc3a WatchSource:0}: Error finding container 1864c33b14f133a9024aa6a23c9aa00ba647cb3cc678448748451568c61ddc3a: Status 404 returned error can't find the container with id 1864c33b14f133a9024aa6a23c9aa00ba647cb3cc678448748451568c61ddc3a
Jul 17 10:59:39.736656 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.736636    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845"
Jul 17 10:59:39.865311 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:39.865286    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55dffc6f4d-gd845"]
Jul 17 10:59:39.868227 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:59:39.868204    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0367d82f_a576_4753_bf69_d536f4a8cfad.slice/crio-a73a0d73790114172e8d914d04b59d84eb30d3efbef246c2b4c6868f5fefb35e WatchSource:0}: Error finding container a73a0d73790114172e8d914d04b59d84eb30d3efbef246c2b4c6868f5fefb35e: Status 404 returned error can't find the container with id a73a0d73790114172e8d914d04b59d84eb30d3efbef246c2b4c6868f5fefb35e
Jul 17 10:59:40.214227 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.214196    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerStarted","Data":"ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232"}
Jul 17 10:59:40.214370 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.214311    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerStarted","Data":"42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8"}
Jul 17 10:59:40.214370 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.214351    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerStarted","Data":"7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152"}
Jul 17 10:59:40.214456 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.214393    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerStarted","Data":"ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9"}
Jul 17 10:59:40.214456 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.214423    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerStarted","Data":"ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0"}
Jul 17 10:59:40.215425 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.215401    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845" event={"ID":"0367d82f-a576-4753-bf69-d536f4a8cfad","Type":"ContainerStarted","Data":"a73a0d73790114172e8d914d04b59d84eb30d3efbef246c2b4c6868f5fefb35e"}
Jul 17 10:59:40.216520 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.216494    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh" event={"ID":"a9d66ff0-9de6-46f8-8742-5e66bf07d950","Type":"ContainerStarted","Data":"1864c33b14f133a9024aa6a23c9aa00ba647cb3cc678448748451568c61ddc3a"}
Jul 17 10:59:40.894268 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.894235    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:59:40.896661 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.896638    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/679aabef-721d-436b-977a-87f5d0608638-metrics-tls\") pod \"dns-default-7hgrl\" (UID: \"679aabef-721d-436b-977a-87f5d0608638\") " pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:59:40.992277 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:40.992241    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w544b\""
Jul 17 10:59:41.000727 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:41.000689    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:59:42.211437 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:42.211409    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7hgrl"]
Jul 17 10:59:42.225675 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:42.225643    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerStarted","Data":"7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8"}
Jul 17 10:59:42.227046 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:42.227019    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh" event={"ID":"a9d66ff0-9de6-46f8-8742-5e66bf07d950","Type":"ContainerStarted","Data":"afb1203583919bd93657af9f55959ab884c95312dd9b09fc5c2e2db416bd4ba6"}
Jul 17 10:59:42.227212 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:42.227193    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:42.231585 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:42.231565    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh"
Jul 17 10:59:42.257090 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:42.257057    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.504200039 podStartE2EDuration="7.25704244s" totalImagesPullingTime="4.735735247s" totalInitContainerRuntime="17.107154ms" isStatefulPod=false podCreationTimestamp="2026-07-17 10:59:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:42.256840767 +0000 UTC m=+165.113924712" watchObservedRunningTime="2026-07-17 10:59:42.25704244 +0000 UTC m=+165.114126385"
Jul 17 10:59:42.274885 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:42.274834    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5fkfh" podStartSLOduration=1.9189316349999999 podStartE2EDuration="4.274816165s" totalImagesPullingTime="2.35588453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:42.27400357 +0000 UTC m=+165.131087518" watchObservedRunningTime="2026-07-17 10:59:42.274816165 +0000 UTC m=+165.131900113"
Jul 17 10:59:42.369950 ip-10-0-134-138 kubenswrapper[2566]: W0717 10:59:42.369884    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod679aabef_721d_436b_977a_87f5d0608638.slice/crio-3f61eced77642523b77e6e72e859f92eea420bbd2402394791a7570870e773d7 WatchSource:0}: Error finding container 3f61eced77642523b77e6e72e859f92eea420bbd2402394791a7570870e773d7: Status 404 returned error can't find the container with id 3f61eced77642523b77e6e72e859f92eea420bbd2402394791a7570870e773d7
Jul 17 10:59:43.234915 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:43.234870    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845" event={"ID":"0367d82f-a576-4753-bf69-d536f4a8cfad","Type":"ContainerStarted","Data":"5fe691448151c253e8f941849bb97da6bd755e6d74bed03f1db7793b33ef779f"}
Jul 17 10:59:43.235363 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:43.234939    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845" event={"ID":"0367d82f-a576-4753-bf69-d536f4a8cfad","Type":"ContainerStarted","Data":"559638a23bc7cd34d661daa647ddd8c24b28e0cf891ab744563c75fc415ed651"}
Jul 17 10:59:43.235363 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:43.234962    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845" event={"ID":"0367d82f-a576-4753-bf69-d536f4a8cfad","Type":"ContainerStarted","Data":"0d1fb05e35cd24037befe7d6544da7b68e2d2f4d25d2c912e8bccb03e19cf157"}
Jul 17 10:59:43.236325 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:43.236298    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hgrl" event={"ID":"679aabef-721d-436b-977a-87f5d0608638","Type":"ContainerStarted","Data":"3f61eced77642523b77e6e72e859f92eea420bbd2402394791a7570870e773d7"}
Jul 17 10:59:43.264608 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:43.264563    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-55dffc6f4d-gd845" podStartSLOduration=1.739880155 podStartE2EDuration="4.264545908s" totalImagesPullingTime="2.524665753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:43.261818113 +0000 UTC m=+166.118902086" watchObservedRunningTime="2026-07-17 10:59:43.264545908 +0000 UTC m=+166.121629853"
Jul 17 10:59:45.245780 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:45.245690    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hgrl" event={"ID":"679aabef-721d-436b-977a-87f5d0608638","Type":"ContainerStarted","Data":"91e366ef8844778f21d30a01e16f096c9c3bfed93cf3295153ef58f13d87b26c"}
Jul 17 10:59:45.246144 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:45.245784    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hgrl" event={"ID":"679aabef-721d-436b-977a-87f5d0608638","Type":"ContainerStarted","Data":"446b5897340408e1356e0a538853f5506894596059ae47726d24a672239010a8"}
Jul 17 10:59:45.246144 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:45.245850    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:59:45.264807 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:45.264765    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7hgrl" podStartSLOduration=131.095335614 podStartE2EDuration="2m13.264750798s" totalImagesPullingTime="2.169415184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:59:45.263402094 +0000 UTC m=+168.120486106" watchObservedRunningTime="2026-07-17 10:59:45.264750798 +0000 UTC m=+168.121834766"
Jul 17 10:59:55.251649 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:55.251575    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7hgrl"
Jul 17 10:59:55.453415 ip-10-0-134-138 kubenswrapper[2566]: I0717 10:59:55.453380    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4d588f69-m4vdd"]
Jul 17 11:00:20.475617 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.475568    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b4d588f69-m4vdd" podUID="5e74e109-9ff6-449b-bb64-9ab2336c450c" containerName="console" containerID="cri-o://3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f" gracePeriod=15
Jul 17 11:00:20.723146 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.723123    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 11:00:20.914327 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914302    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-serving-cert\") pod \"5e74e109-9ff6-449b-bb64-9ab2336c450c\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") "
Jul 17 11:00:20.914513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914333    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-oauth-serving-cert\") pod \"5e74e109-9ff6-449b-bb64-9ab2336c450c\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") "
Jul 17 11:00:20.914513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914364    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-trusted-ca-bundle\") pod \"5e74e109-9ff6-449b-bb64-9ab2336c450c\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") "
Jul 17 11:00:20.914513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914398    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-oauth-config\") pod \"5e74e109-9ff6-449b-bb64-9ab2336c450c\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") "
Jul 17 11:00:20.914680 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914512    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-service-ca\") pod \"5e74e109-9ff6-449b-bb64-9ab2336c450c\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") "
Jul 17 11:00:20.914680 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914577    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzz67\" (UniqueName: \"kubernetes.io/projected/5e74e109-9ff6-449b-bb64-9ab2336c450c-kube-api-access-xzz67\") pod \"5e74e109-9ff6-449b-bb64-9ab2336c450c\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") "
Jul 17 11:00:20.914680 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914626    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-config\") pod \"5e74e109-9ff6-449b-bb64-9ab2336c450c\" (UID: \"5e74e109-9ff6-449b-bb64-9ab2336c450c\") "
Jul 17 11:00:20.914868 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914766    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5e74e109-9ff6-449b-bb64-9ab2336c450c" (UID: "5e74e109-9ff6-449b-bb64-9ab2336c450c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:00:20.914933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914809    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5e74e109-9ff6-449b-bb64-9ab2336c450c" (UID: "5e74e109-9ff6-449b-bb64-9ab2336c450c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:00:20.915004 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.914978    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-service-ca" (OuterVolumeSpecName: "service-ca") pod "5e74e109-9ff6-449b-bb64-9ab2336c450c" (UID: "5e74e109-9ff6-449b-bb64-9ab2336c450c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:00:20.915063 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.915029    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-oauth-serving-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:20.915063 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.915042    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-trusted-ca-bundle\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:20.915164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.915072    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-config" (OuterVolumeSpecName: "console-config") pod "5e74e109-9ff6-449b-bb64-9ab2336c450c" (UID: "5e74e109-9ff6-449b-bb64-9ab2336c450c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:00:20.916685 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.916654    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5e74e109-9ff6-449b-bb64-9ab2336c450c" (UID: "5e74e109-9ff6-449b-bb64-9ab2336c450c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:00:20.916685 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.916670    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5e74e109-9ff6-449b-bb64-9ab2336c450c" (UID: "5e74e109-9ff6-449b-bb64-9ab2336c450c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:00:20.916840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:20.916717    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e74e109-9ff6-449b-bb64-9ab2336c450c-kube-api-access-xzz67" (OuterVolumeSpecName: "kube-api-access-xzz67") pod "5e74e109-9ff6-449b-bb64-9ab2336c450c" (UID: "5e74e109-9ff6-449b-bb64-9ab2336c450c"). InnerVolumeSpecName "kube-api-access-xzz67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:00:21.016368 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.016337    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-config\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:21.016368 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.016363    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-serving-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:21.016368 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.016372    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e74e109-9ff6-449b-bb64-9ab2336c450c-console-oauth-config\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:21.016554 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.016383    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e74e109-9ff6-449b-bb64-9ab2336c450c-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:21.016554 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.016392    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzz67\" (UniqueName: \"kubernetes.io/projected/5e74e109-9ff6-449b-bb64-9ab2336c450c-kube-api-access-xzz67\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:21.379676 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.379641    2566 generic.go:356] "Generic (PLEG): container finished" podID="5e74e109-9ff6-449b-bb64-9ab2336c450c" containerID="3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f" exitCode=0
Jul 17 11:00:21.379877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.379690    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4d588f69-m4vdd"
Jul 17 11:00:21.379877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.379738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4d588f69-m4vdd" event={"ID":"5e74e109-9ff6-449b-bb64-9ab2336c450c","Type":"ContainerDied","Data":"3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f"}
Jul 17 11:00:21.379877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.379789    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4d588f69-m4vdd" event={"ID":"5e74e109-9ff6-449b-bb64-9ab2336c450c","Type":"ContainerDied","Data":"94a74c07ae16b73d63d5069e02d28394fa08e463601e07ae906cb36db3b01c18"}
Jul 17 11:00:21.379877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.379808    2566 scope.go:117] "RemoveContainer" containerID="3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f"
Jul 17 11:00:21.388551 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.388532    2566 scope.go:117] "RemoveContainer" containerID="3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f"
Jul 17 11:00:21.388827 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:00:21.388808    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f\": container with ID starting with 3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f not found: ID does not exist" containerID="3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f"
Jul 17 11:00:21.388887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.388835    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f"} err="failed to get container status \"3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f\": rpc error: code = NotFound desc = could not find container \"3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f\": container with ID starting with 3c38b9ff6454b7a7657c649880346b39bdbedee0754757710ffc82a068f08f0f not found: ID does not exist"
Jul 17 11:00:21.402674 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.402652    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4d588f69-m4vdd"]
Jul 17 11:00:21.407196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.407170    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b4d588f69-m4vdd"]
Jul 17 11:00:21.657663 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:21.657564    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e74e109-9ff6-449b-bb64-9ab2336c450c" path="/var/lib/kubelet/pods/5e74e109-9ff6-449b-bb64-9ab2336c450c/volumes"
Jul 17 11:00:54.767302 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:54.767260    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 11:00:54.767853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:54.767542    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="alertmanager" containerID="cri-o://ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0" gracePeriod=120
Jul 17 11:00:54.767853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:54.767594    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy-metric" containerID="cri-o://ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232" gracePeriod=120
Jul 17 11:00:54.767853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:54.767631    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy" containerID="cri-o://42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8" gracePeriod=120
Jul 17 11:00:54.767853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:54.767619    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy-web" containerID="cri-o://7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152" gracePeriod=120
Jul 17 11:00:54.767853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:54.767651    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="config-reloader" containerID="cri-o://ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9" gracePeriod=120
Jul 17 11:00:54.767853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:54.767748    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="prom-label-proxy" containerID="cri-o://7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8" gracePeriod=120
Jul 17 11:00:55.505774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:55.505736    2566 generic.go:356] "Generic (PLEG): container finished" podID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerID="7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8" exitCode=0
Jul 17 11:00:55.505774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:55.505762    2566 generic.go:356] "Generic (PLEG): container finished" podID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerID="42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8" exitCode=0
Jul 17 11:00:55.505774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:55.505771    2566 generic.go:356] "Generic (PLEG): container finished" podID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerID="ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9" exitCode=0
Jul 17 11:00:55.505774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:55.505779    2566 generic.go:356] "Generic (PLEG): container finished" podID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerID="ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0" exitCode=0
Jul 17 11:00:55.506038 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:55.505812    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerDied","Data":"7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8"}
Jul 17 11:00:55.506038 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:55.505899    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerDied","Data":"42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8"}
Jul 17 11:00:55.506038 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:55.505926    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerDied","Data":"ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9"}
Jul 17 11:00:55.506038 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:55.505951    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerDied","Data":"ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0"}
Jul 17 11:00:56.001266 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.001243    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.057991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.057963    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058001    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-trusted-ca-bundle\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058029    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-tls-assets\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058053    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-main-db\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058088    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-main-tls\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058107    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-cluster-tls-config\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058140    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-web-config\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058168    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-volume\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058193    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9lt5\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-kube-api-access-r9lt5\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058239    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-out\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058261    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058311    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058334    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-metrics-client-ca\") pod \"6fd75c4e-9cc4-482e-ae32-04da804666bb\" (UID: \"6fd75c4e-9cc4-482e-ae32-04da804666bb\") "
Jul 17 11:00:56.058791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058615    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:00:56.059125 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.058934    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:00:56.059359 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.059315    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:00:56.061870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.061831    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-out" (OuterVolumeSpecName: "config-out") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:00:56.061975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.061882    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-kube-api-access-r9lt5" (OuterVolumeSpecName: "kube-api-access-r9lt5") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "kube-api-access-r9lt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:00:56.061975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.061900    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:00:56.062094 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.061981    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:00:56.062094 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.062065    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-volume" (OuterVolumeSpecName: "config-volume") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:00:56.063017 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.062996    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:00:56.063222 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.063200    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:00:56.063747 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.063685    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:00:56.066243 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.066189    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:00:56.071126 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.071104    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-web-config" (OuterVolumeSpecName: "web-config") pod "6fd75c4e-9cc4-482e-ae32-04da804666bb" (UID: "6fd75c4e-9cc4-482e-ae32-04da804666bb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:00:56.159302 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159271    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159302 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159296    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159302 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159309    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-metrics-client-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159319    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159329    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159339    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-tls-assets\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159346    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-alertmanager-main-db\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159355    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-secret-alertmanager-main-tls\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159363    2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-cluster-tls-config\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159371    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-web-config\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159379    2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-volume\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159386    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9lt5\" (UniqueName: \"kubernetes.io/projected/6fd75c4e-9cc4-482e-ae32-04da804666bb-kube-api-access-r9lt5\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.159518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.159393    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fd75c4e-9cc4-482e-ae32-04da804666bb-config-out\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:00:56.512387 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.512307    2566 generic.go:356] "Generic (PLEG): container finished" podID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerID="ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232" exitCode=0
Jul 17 11:00:56.512387 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.512330    2566 generic.go:356] "Generic (PLEG): container finished" podID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerID="7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152" exitCode=0
Jul 17 11:00:56.512387 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.512368    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerDied","Data":"ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232"}
Jul 17 11:00:56.512645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.512399    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.512645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.512426    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerDied","Data":"7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152"}
Jul 17 11:00:56.512645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.512452    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fd75c4e-9cc4-482e-ae32-04da804666bb","Type":"ContainerDied","Data":"f948b24eee581156c44331ff83379dfb6b5a700c5c947277f0b7ba09c33c54dd"}
Jul 17 11:00:56.512645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.512483    2566 scope.go:117] "RemoveContainer" containerID="7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8"
Jul 17 11:00:56.520009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.519901    2566 scope.go:117] "RemoveContainer" containerID="ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232"
Jul 17 11:00:56.526614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.526599    2566 scope.go:117] "RemoveContainer" containerID="42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8"
Jul 17 11:00:56.532635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.532620    2566 scope.go:117] "RemoveContainer" containerID="7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152"
Jul 17 11:00:56.538472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.538456    2566 scope.go:117] "RemoveContainer" containerID="ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9"
Jul 17 11:00:56.544739 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.544692    2566 scope.go:117] "RemoveContainer" containerID="ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0"
Jul 17 11:00:56.551288 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.551272    2566 scope.go:117] "RemoveContainer" containerID="f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2"
Jul 17 11:00:56.551343 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.551272    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 11:00:56.557500 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.557485    2566 scope.go:117] "RemoveContainer" containerID="7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8"
Jul 17 11:00:56.557752 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:00:56.557736    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8\": container with ID starting with 7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8 not found: ID does not exist" containerID="7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8"
Jul 17 11:00:56.557792 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.557760    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8"} err="failed to get container status \"7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8\": rpc error: code = NotFound desc = could not find container \"7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8\": container with ID starting with 7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8 not found: ID does not exist"
Jul 17 11:00:56.557792 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.557776    2566 scope.go:117] "RemoveContainer" containerID="ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232"
Jul 17 11:00:56.558001 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:00:56.557986    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232\": container with ID starting with ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232 not found: ID does not exist" containerID="ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232"
Jul 17 11:00:56.558051 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.558004    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232"} err="failed to get container status \"ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232\": rpc error: code = NotFound desc = could not find container \"ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232\": container with ID starting with ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232 not found: ID does not exist"
Jul 17 11:00:56.558051 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.558018    2566 scope.go:117] "RemoveContainer" containerID="42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8"
Jul 17 11:00:56.558208 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:00:56.558192    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8\": container with ID starting with 42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8 not found: ID does not exist" containerID="42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8"
Jul 17 11:00:56.558242 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.558211    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8"} err="failed to get container status \"42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8\": rpc error: code = NotFound desc = could not find container \"42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8\": container with ID starting with 42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8 not found: ID does not exist"
Jul 17 11:00:56.558242 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.558223    2566 scope.go:117] "RemoveContainer" containerID="7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152"
Jul 17 11:00:56.558460 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:00:56.558442    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152\": container with ID starting with 7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152 not found: ID does not exist" containerID="7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152"
Jul 17 11:00:56.558507 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.558466    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152"} err="failed to get container status \"7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152\": rpc error: code = NotFound desc = could not find container \"7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152\": container with ID starting with 7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152 not found: ID does not exist"
Jul 17 11:00:56.558507 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.558487    2566 scope.go:117] "RemoveContainer" containerID="ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9"
Jul 17 11:00:56.558785 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:00:56.558759    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9\": container with ID starting with ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9 not found: ID does not exist" containerID="ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9"
Jul 17 11:00:56.558887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.558789    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9"} err="failed to get container status \"ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9\": rpc error: code = NotFound desc = could not find container \"ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9\": container with ID starting with ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9 not found: ID does not exist"
Jul 17 11:00:56.558887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.558809    2566 scope.go:117] "RemoveContainer" containerID="ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0"
Jul 17 11:00:56.559067 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:00:56.559008    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0\": container with ID starting with ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0 not found: ID does not exist" containerID="ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0"
Jul 17 11:00:56.559144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.559062    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0"} err="failed to get container status \"ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0\": rpc error: code = NotFound desc = could not find container \"ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0\": container with ID starting with ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0 not found: ID does not exist"
Jul 17 11:00:56.559144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.559083    2566 scope.go:117] "RemoveContainer" containerID="f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2"
Jul 17 11:00:56.559528 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:00:56.559427    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2\": container with ID starting with f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2 not found: ID does not exist" containerID="f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2"
Jul 17 11:00:56.559528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.559456    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2"} err="failed to get container status \"f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2\": rpc error: code = NotFound desc = could not find container \"f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2\": container with ID starting with f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2 not found: ID does not exist"
Jul 17 11:00:56.559528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.559477    2566 scope.go:117] "RemoveContainer" containerID="7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8"
Jul 17 11:00:56.559772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.559754    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8"} err="failed to get container status \"7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8\": rpc error: code = NotFound desc = could not find container \"7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8\": container with ID starting with 7678b1238844f18ea535a33594b98036b4b857d37baf7db2082c19d344bb1ea8 not found: ID does not exist"
Jul 17 11:00:56.559829 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.559772    2566 scope.go:117] "RemoveContainer" containerID="ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232"
Jul 17 11:00:56.560028 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560007    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232"} err="failed to get container status \"ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232\": rpc error: code = NotFound desc = could not find container \"ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232\": container with ID starting with ad79763d36ecdc552d156227794e58eaea976c6ff547ad3876ab18498f6d0232 not found: ID does not exist"
Jul 17 11:00:56.560098 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560032    2566 scope.go:117] "RemoveContainer" containerID="42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8"
Jul 17 11:00:56.560264 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560245    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8"} err="failed to get container status \"42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8\": rpc error: code = NotFound desc = could not find container \"42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8\": container with ID starting with 42ded4f38e8b9fcdddc0467645d565d7c46f6e55eb586e49ae7b4f472d541bf8 not found: ID does not exist"
Jul 17 11:00:56.560312 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560265    2566 scope.go:117] "RemoveContainer" containerID="7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152"
Jul 17 11:00:56.560506 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560487    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152"} err="failed to get container status \"7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152\": rpc error: code = NotFound desc = could not find container \"7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152\": container with ID starting with 7fd32fb18d6acd60257b3dcf862736f99aac0ed8de3b77e0d85cf98e57e19152 not found: ID does not exist"
Jul 17 11:00:56.560583 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560508    2566 scope.go:117] "RemoveContainer" containerID="ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9"
Jul 17 11:00:56.560752 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560734    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9"} err="failed to get container status \"ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9\": rpc error: code = NotFound desc = could not find container \"ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9\": container with ID starting with ffec7838f4d1a6f202526b5c529ceab583f6d40715156478fd67d52e1f1883a9 not found: ID does not exist"
Jul 17 11:00:56.560817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560753    2566 scope.go:117] "RemoveContainer" containerID="ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0"
Jul 17 11:00:56.560941 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560924    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0"} err="failed to get container status \"ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0\": rpc error: code = NotFound desc = could not find container \"ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0\": container with ID starting with ea89c3fe8d6820b2b88688d2ecaad2a0ad13d8272293b3d19f45273b8f82acf0 not found: ID does not exist"
Jul 17 11:00:56.561009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.560942    2566 scope.go:117] "RemoveContainer" containerID="f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2"
Jul 17 11:00:56.561168 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.561151    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2"} err="failed to get container status \"f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2\": rpc error: code = NotFound desc = could not find container \"f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2\": container with ID starting with f77cba140b854b5bc90b8afb28c5d222fba605675e4b88bb690423ae90b031e2 not found: ID does not exist"
Jul 17 11:00:56.561214 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.561188    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 11:00:56.595498 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595473    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 11:00:56.595723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595696    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="config-reloader"
Jul 17 11:00:56.595768 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595724    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="config-reloader"
Jul 17 11:00:56.595768 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595732    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="prom-label-proxy"
Jul 17 11:00:56.595768 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595738    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="prom-label-proxy"
Jul 17 11:00:56.595768 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595746    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="alertmanager"
Jul 17 11:00:56.595768 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595752    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="alertmanager"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595758    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e74e109-9ff6-449b-bb64-9ab2336c450c" containerName="console"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595779    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e74e109-9ff6-449b-bb64-9ab2336c450c" containerName="console"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595786    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy-metric"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595791    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy-metric"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595801    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy-web"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595806    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy-web"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595812    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595817    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595825    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="init-config-reloader"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595830    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="init-config-reloader"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595870    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="config-reloader"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595877    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="alertmanager"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595883    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595889    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy-web"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595895    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="prom-label-proxy"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595900    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e74e109-9ff6-449b-bb64-9ab2336c450c" containerName="console"
Jul 17 11:00:56.595910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.595904    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" containerName="kube-rbac-proxy-metric"
Jul 17 11:00:56.600548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.600534    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.626387 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.626364    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 11:00:56.628041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.628025    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 17 11:00:56.629075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.629054    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 17 11:00:56.629238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.629221    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 17 11:00:56.629328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.629305    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 17 11:00:56.629626 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.629610    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 17 11:00:56.629752 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.629735    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 17 11:00:56.629996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.629982    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 17 11:00:56.630815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.630798    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 17 11:00:56.631535 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.631515    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wfjxm\""
Jul 17 11:00:56.637586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.637570    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 17 11:00:56.661803 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.661785    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.661893 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.661811    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.661893 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.661830    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-config-volume\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.661893 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.661845    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.661893 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.661861    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.662031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.661895    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.662031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.661917    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.662031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.661950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-web-config\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.662031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.661989    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.662031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.662027    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffwh6\" (UniqueName: \"kubernetes.io/projected/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-kube-api-access-ffwh6\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.662176 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.662050    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.662176 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.662089    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-config-out\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.662176 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.662105    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.762770 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.762717    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-web-config\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.762770 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.762748    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.762901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.762782    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffwh6\" (UniqueName: \"kubernetes.io/projected/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-kube-api-access-ffwh6\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.762901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.762810    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.762901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.762865    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-config-out\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.762901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.762889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.763088 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.762933    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.763088 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.762960    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.763088 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.762986    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-config-volume\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.763088 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.763012    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.763088 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.763034    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.763088 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.763065    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.763435 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.763089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.763435 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.763281    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.764795 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.764769    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.765357 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.765330    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.765834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.765790    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-config-volume\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.765834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.765790    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.765973 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.765874    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.766057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.766039    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-web-config\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.766196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.766177    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.766342 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.766326    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.766415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.766393    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.767267 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.767249    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-config-out\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.767719 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.767689    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.772935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.772918    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffwh6\" (UniqueName: \"kubernetes.io/projected/6b8ab507-0b7d-407a-8cfd-c72368a0ba02-kube-api-access-ffwh6\") pod \"alertmanager-main-0\" (UID: \"6b8ab507-0b7d-407a-8cfd-c72368a0ba02\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:56.908719 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:56.908681    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 11:00:57.040152 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:57.040127    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 11:00:57.042163 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:00:57.042138    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8ab507_0b7d_407a_8cfd_c72368a0ba02.slice/crio-c7ddd673d82c26dfbb2a434349ccc7adc9c133133f964b48ac514314b7fe09c2 WatchSource:0}: Error finding container c7ddd673d82c26dfbb2a434349ccc7adc9c133133f964b48ac514314b7fe09c2: Status 404 returned error can't find the container with id c7ddd673d82c26dfbb2a434349ccc7adc9c133133f964b48ac514314b7fe09c2
Jul 17 11:00:57.517966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:57.517928    2566 generic.go:356] "Generic (PLEG): container finished" podID="6b8ab507-0b7d-407a-8cfd-c72368a0ba02" containerID="bb46fe730e79c0a74703a58f333c746344bf07856dfd3e5129e9f6101a662090" exitCode=0
Jul 17 11:00:57.518105 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:57.518005    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b8ab507-0b7d-407a-8cfd-c72368a0ba02","Type":"ContainerDied","Data":"bb46fe730e79c0a74703a58f333c746344bf07856dfd3e5129e9f6101a662090"}
Jul 17 11:00:57.518105 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:57.518071    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b8ab507-0b7d-407a-8cfd-c72368a0ba02","Type":"ContainerStarted","Data":"c7ddd673d82c26dfbb2a434349ccc7adc9c133133f964b48ac514314b7fe09c2"}
Jul 17 11:00:57.656272 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:57.656248    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fd75c4e-9cc4-482e-ae32-04da804666bb" path="/var/lib/kubelet/pods/6fd75c4e-9cc4-482e-ae32-04da804666bb/volumes"
Jul 17 11:00:58.524990 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:58.524957    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b8ab507-0b7d-407a-8cfd-c72368a0ba02","Type":"ContainerStarted","Data":"af5011ffb9c0d9f50a71b25b659f60c2a7ac7d71778fe59914f00f53c6bf98c4"}
Jul 17 11:00:58.525371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:58.525018    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b8ab507-0b7d-407a-8cfd-c72368a0ba02","Type":"ContainerStarted","Data":"8bb84bd92d56e296e11327ac6a0dfdcfe8f2968de85fae30e47151217fce05ec"}
Jul 17 11:00:58.525371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:58.525045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b8ab507-0b7d-407a-8cfd-c72368a0ba02","Type":"ContainerStarted","Data":"b57e923b38b4d7297167aee648d52e7f0bf5b9c00f3740c4c134ba866b90355c"}
Jul 17 11:00:58.525371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:58.525069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b8ab507-0b7d-407a-8cfd-c72368a0ba02","Type":"ContainerStarted","Data":"2020e21da8139b5f73d4296d80732079e673a0352bebdbaa4de25eb29cea2ef2"}
Jul 17 11:00:58.525371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:58.525093    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b8ab507-0b7d-407a-8cfd-c72368a0ba02","Type":"ContainerStarted","Data":"df9a86f951c0b3788a9a8c8507ee8421b86c1ba532dfafc1bd53c1c86989d630"}
Jul 17 11:00:58.525371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:58.525123    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6b8ab507-0b7d-407a-8cfd-c72368a0ba02","Type":"ContainerStarted","Data":"cc81604fba5ed70838fd6d4643417220ff109420c0237a07c0a56639d41f6420"}
Jul 17 11:00:58.556145 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:00:58.556092    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.541733523 podStartE2EDuration="2.556075691s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.342168ms" isStatefulPod=false podCreationTimestamp="2026-07-17 11:00:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:00:58.553669844 +0000 UTC m=+241.410753789" watchObservedRunningTime="2026-07-17 11:00:58.556075691 +0000 UTC m=+241.413160039"
Jul 17 11:01:01.256302 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.256264    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7459678d7d-x675z"]
Jul 17 11:01:01.259668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.259642    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.267693 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.267672    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 11:01:01.267934 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.267914    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 11:01:01.267991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.267914    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tnv9t\""
Jul 17 11:01:01.268817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.268791    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 11:01:01.268923 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.268836    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 11:01:01.269143 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.269118    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 11:01:01.269255 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.269200    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 11:01:01.270414 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.270390    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 11:01:01.273056 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.273038    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 11:01:01.276467 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.276444    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7459678d7d-x675z"]
Jul 17 11:01:01.292602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.292580    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjjwd\" (UniqueName: \"kubernetes.io/projected/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-kube-api-access-jjjwd\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.292698 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.292623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-oauth-config\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.292698 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.292646    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-trusted-ca-bundle\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.292801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.292745    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-oauth-serving-cert\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.292801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.292775    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-serving-cert\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.292801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.292791    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-service-ca\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.292932 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.292834    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-config\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.393934 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.393902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-trusted-ca-bundle\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394068 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.393943    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-oauth-serving-cert\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394068 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.393966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-serving-cert\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394068 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.393988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-service-ca\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394068 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.394021    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-config\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394068 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.394063    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjjwd\" (UniqueName: \"kubernetes.io/projected/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-kube-api-access-jjjwd\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394315 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.394111    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-oauth-config\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394725 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.394685    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-oauth-serving-cert\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394884 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.394865    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-trusted-ca-bundle\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394936 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.394867    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-config\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.394936 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.394869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-service-ca\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.397009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.396985    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-serving-cert\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.397121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.397102    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-oauth-config\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.403525 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.403500    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjjwd\" (UniqueName: \"kubernetes.io/projected/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-kube-api-access-jjjwd\") pod \"console-7459678d7d-x675z\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") " pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.568687 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.568611    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:01.696324 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:01.696292    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7459678d7d-x675z"]
Jul 17 11:01:01.699453 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:01:01.699424    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd1be43_7c10_4b27_8de0_aa9e1834dd61.slice/crio-83f0cb3f626bb47f68a9dc0bf478736c5a0c77c5d9c28154fd216391b04a234e WatchSource:0}: Error finding container 83f0cb3f626bb47f68a9dc0bf478736c5a0c77c5d9c28154fd216391b04a234e: Status 404 returned error can't find the container with id 83f0cb3f626bb47f68a9dc0bf478736c5a0c77c5d9c28154fd216391b04a234e
Jul 17 11:01:02.542428 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:02.542395    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7459678d7d-x675z" event={"ID":"bfd1be43-7c10-4b27-8de0-aa9e1834dd61","Type":"ContainerStarted","Data":"d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d"}
Jul 17 11:01:02.542803 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:02.542443    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7459678d7d-x675z" event={"ID":"bfd1be43-7c10-4b27-8de0-aa9e1834dd61","Type":"ContainerStarted","Data":"83f0cb3f626bb47f68a9dc0bf478736c5a0c77c5d9c28154fd216391b04a234e"}
Jul 17 11:01:02.563302 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:02.563254    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7459678d7d-x675z" podStartSLOduration=1.5632409950000001 podStartE2EDuration="1.563240995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:01:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:01:02.56237383 +0000 UTC m=+245.419457774" watchObservedRunningTime="2026-07-17 11:01:02.563240995 +0000 UTC m=+245.420324941"
Jul 17 11:01:11.569391 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:11.569353    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:11.569803 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:11.569440    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:11.574189 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:11.574165    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:11.578486 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:11.578466    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:01:51.330047 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.330012    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"]
Jul 17 11:01:51.333110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.333095    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.337501 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.337476    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 17 11:01:51.337635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.337533    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 17 11:01:51.337635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.337493    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 17 11:01:51.337635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.337581    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 17 11:01:51.337844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.337546    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 17 11:01:51.337844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.337515    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 17 11:01:51.337844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.337552    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 17 11:01:51.345844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.345823    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"]
Jul 17 11:01:51.455774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.455742    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/828f3c98-6496-47cf-8439-9a0cc5e7474d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.455974 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.455786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-ca\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.455974 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.455808    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.455974 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.455890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z5tk\" (UniqueName: \"kubernetes.io/projected/828f3c98-6496-47cf-8439-9a0cc5e7474d-kube-api-access-8z5tk\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.456137 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.455971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-hub\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.456137 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.456012    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.557226 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.557191    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-ca\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.557394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.557234    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.557394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.557263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z5tk\" (UniqueName: \"kubernetes.io/projected/828f3c98-6496-47cf-8439-9a0cc5e7474d-kube-api-access-8z5tk\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.557394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.557300    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-hub\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.557394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.557327    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.557394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.557388    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/828f3c98-6496-47cf-8439-9a0cc5e7474d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.558146 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.558116    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/828f3c98-6496-47cf-8439-9a0cc5e7474d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.559784 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.559758    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-hub\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.559876 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.559812    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.559919 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.559892    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-ca\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.559967 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.559950    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/828f3c98-6496-47cf-8439-9a0cc5e7474d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.567411 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.567386    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z5tk\" (UniqueName: \"kubernetes.io/projected/828f3c98-6496-47cf-8439-9a0cc5e7474d-kube-api-access-8z5tk\") pod \"cluster-proxy-proxy-agent-84fb778dc5-jrln9\" (UID: \"828f3c98-6496-47cf-8439-9a0cc5e7474d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.652640 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.652575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"
Jul 17 11:01:51.788640 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:51.788558    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9"]
Jul 17 11:01:51.791360 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:01:51.791329    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod828f3c98_6496_47cf_8439_9a0cc5e7474d.slice/crio-b575c9efcb482f97cdf851e24027bf6fa2976083d6604069fbc9750a55b7f12d WatchSource:0}: Error finding container b575c9efcb482f97cdf851e24027bf6fa2976083d6604069fbc9750a55b7f12d: Status 404 returned error can't find the container with id b575c9efcb482f97cdf851e24027bf6fa2976083d6604069fbc9750a55b7f12d
Jul 17 11:01:52.715047 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:52.715001    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9" event={"ID":"828f3c98-6496-47cf-8439-9a0cc5e7474d","Type":"ContainerStarted","Data":"b575c9efcb482f97cdf851e24027bf6fa2976083d6604069fbc9750a55b7f12d"}
Jul 17 11:01:55.729106 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:55.729074    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9" event={"ID":"828f3c98-6496-47cf-8439-9a0cc5e7474d","Type":"ContainerStarted","Data":"c7cd20b967753f470c836f691475a60eae49788398bf3e50f612ec5a269f0378"}
Jul 17 11:01:55.729461 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:55.729123    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9" event={"ID":"828f3c98-6496-47cf-8439-9a0cc5e7474d","Type":"ContainerStarted","Data":"3b6476dd2a8ce7c989bb02eef68f192537c4c873eb7d1c77f1adda1d84cd1a6d"}
Jul 17 11:01:55.729461 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:55.729145    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9" event={"ID":"828f3c98-6496-47cf-8439-9a0cc5e7474d","Type":"ContainerStarted","Data":"75305d559245b754fc17aa46eba573ef57a760cf58b22863628e8970941f4cde"}
Jul 17 11:01:55.749695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:55.749649    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84fb778dc5-jrln9" podStartSLOduration=1.2298146349999999 podStartE2EDuration="4.749633395s" totalImagesPullingTime="3.51981876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:01:55.749308371 +0000 UTC m=+298.606392343" watchObservedRunningTime="2026-07-17 11:01:55.749633395 +0000 UTC m=+298.606717341"
Jul 17 11:01:57.578380 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:57.578353    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:01:57.581418 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:57.581395    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:01:57.586501 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:57.586478    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:01:57.589747 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:57.589725    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 11:01:57.591117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:01:57.591100    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:02:12.798565 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:12.798528    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"]
Jul 17 11:02:12.830906 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:12.830877    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"]
Jul 17 11:02:12.831045 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:12.830987    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"
Jul 17 11:02:12.835000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:12.834974    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-jz98r\""
Jul 17 11:02:12.835243 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:12.835225    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 17 11:02:12.835311 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:12.835225    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 17 11:02:12.926876 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:12.926833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ztkn\" (UniqueName: \"kubernetes.io/projected/b6916e19-ad3c-4bf0-8fdc-d344422fe2fc-kube-api-access-6ztkn\") pod \"cert-manager-operator-controller-manager-55c444bf86-lzqlf\" (UID: \"b6916e19-ad3c-4bf0-8fdc-d344422fe2fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"
Jul 17 11:02:12.926876 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:12.926879    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6916e19-ad3c-4bf0-8fdc-d344422fe2fc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lzqlf\" (UID: \"b6916e19-ad3c-4bf0-8fdc-d344422fe2fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"
Jul 17 11:02:13.028041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:13.028012    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ztkn\" (UniqueName: \"kubernetes.io/projected/b6916e19-ad3c-4bf0-8fdc-d344422fe2fc-kube-api-access-6ztkn\") pod \"cert-manager-operator-controller-manager-55c444bf86-lzqlf\" (UID: \"b6916e19-ad3c-4bf0-8fdc-d344422fe2fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"
Jul 17 11:02:13.028192 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:13.028052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6916e19-ad3c-4bf0-8fdc-d344422fe2fc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lzqlf\" (UID: \"b6916e19-ad3c-4bf0-8fdc-d344422fe2fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"
Jul 17 11:02:13.028472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:13.028453    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6916e19-ad3c-4bf0-8fdc-d344422fe2fc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lzqlf\" (UID: \"b6916e19-ad3c-4bf0-8fdc-d344422fe2fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"
Jul 17 11:02:13.037753 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:13.037726    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ztkn\" (UniqueName: \"kubernetes.io/projected/b6916e19-ad3c-4bf0-8fdc-d344422fe2fc-kube-api-access-6ztkn\") pod \"cert-manager-operator-controller-manager-55c444bf86-lzqlf\" (UID: \"b6916e19-ad3c-4bf0-8fdc-d344422fe2fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"
Jul 17 11:02:13.140159 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:13.140128    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"
Jul 17 11:02:13.292843 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:13.292821    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf"]
Jul 17 11:02:13.301288 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:13.301268    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 11:02:13.793447 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:13.793412    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf" event={"ID":"b6916e19-ad3c-4bf0-8fdc-d344422fe2fc","Type":"ContainerStarted","Data":"5e82889fdc06230f6df1322769cb5e53ee96d28f61681b5f07ef4b489a2cc693"}
Jul 17 11:02:15.806071 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:15.806037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf" event={"ID":"b6916e19-ad3c-4bf0-8fdc-d344422fe2fc","Type":"ContainerStarted","Data":"b8b9ce0e438bc83520ee77962fb140e530e70d457fef129f3b495fa4f2b7d54a"}
Jul 17 11:02:15.832042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:15.831982    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lzqlf" podStartSLOduration=1.510661901 podStartE2EDuration="3.831963068s" totalImagesPullingTime="2.321301167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:02:15.830727106 +0000 UTC m=+318.687811049" watchObservedRunningTime="2026-07-17 11:02:15.831963068 +0000 UTC m=+318.689047014"
Jul 17 11:02:17.099351 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:17.099315    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-plbgb"]
Jul 17 11:02:17.101877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:17.101860    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:17.108639 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:02:17.108612    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"cert-manager\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 17 11:02:17.108779 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:02:17.108745    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"cert-manager-webhook-6486ffd997-plbgb\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"cert-manager\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5" pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:17.108779 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:02:17.108761    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cert-manager-webhook-dockercfg-sbl29\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"cert-manager\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sbl29\"" type="*v1.Secret"
Jul 17 11:02:17.121129 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:02:17.121101    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"cert-manager\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 17 11:02:17.135173 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:17.135148    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-plbgb"]
Jul 17 11:02:17.166221 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:17.166194    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-plbgb\" (UID: \"9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:17.166336 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:17.166228    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jjkw\" (UniqueName: \"kubernetes.io/projected/9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5-kube-api-access-2jjkw\") pod \"cert-manager-webhook-6486ffd997-plbgb\" (UID: \"9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:17.267119 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:17.267084    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-plbgb\" (UID: \"9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:17.267119 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:17.267121    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jjkw\" (UniqueName: \"kubernetes.io/projected/9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5-kube-api-access-2jjkw\") pod \"cert-manager-webhook-6486ffd997-plbgb\" (UID: \"9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:17.278824 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:17.278792    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-plbgb\" (UID: \"9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:18.079300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:18.079268    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sbl29\""
Jul 17 11:02:18.276457 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:02:18.276409    2566 projected.go:291] Couldn't get configMap cert-manager/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 17 11:02:18.365471 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:18.365385    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 17 11:02:18.497273 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:18.497240    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 17 11:02:18.506847 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:02:18.506823    2566 projected.go:196] Error preparing data for projected volume kube-api-access-2jjkw for pod cert-manager/cert-manager-webhook-6486ffd997-plbgb: failed to sync configmap cache: timed out waiting for the condition
Jul 17 11:02:18.506961 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:02:18.506889    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5-kube-api-access-2jjkw podName:9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5 nodeName:}" failed. No retries permitted until 2026-07-17 11:02:19.006870981 +0000 UTC m=+321.863954903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2jjkw" (UniqueName: "kubernetes.io/projected/9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5-kube-api-access-2jjkw") pod "cert-manager-webhook-6486ffd997-plbgb" (UID: "9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5") : failed to sync configmap cache: timed out waiting for the condition
Jul 17 11:02:19.080427 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:19.080384    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jjkw\" (UniqueName: \"kubernetes.io/projected/9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5-kube-api-access-2jjkw\") pod \"cert-manager-webhook-6486ffd997-plbgb\" (UID: \"9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:19.082849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:19.082827    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jjkw\" (UniqueName: \"kubernetes.io/projected/9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5-kube-api-access-2jjkw\") pod \"cert-manager-webhook-6486ffd997-plbgb\" (UID: \"9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:19.210787 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:19.210751    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:19.344582 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:19.344554    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-plbgb"]
Jul 17 11:02:19.346906 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:02:19.346872    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9149a9c2_8ab9_420f_9bfe_238e2b2ca3f5.slice/crio-f5ec85522a6551da26514fc075ed89629bbd74508c5e5d69eeb98d6ff828d5cb WatchSource:0}: Error finding container f5ec85522a6551da26514fc075ed89629bbd74508c5e5d69eeb98d6ff828d5cb: Status 404 returned error can't find the container with id f5ec85522a6551da26514fc075ed89629bbd74508c5e5d69eeb98d6ff828d5cb
Jul 17 11:02:19.823576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:19.823539    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb" event={"ID":"9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5","Type":"ContainerStarted","Data":"f5ec85522a6551da26514fc075ed89629bbd74508c5e5d69eeb98d6ff828d5cb"}
Jul 17 11:02:21.833885 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:21.833852    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb" event={"ID":"9149a9c2-8ab9-420f-9bfe-238e2b2ca3f5","Type":"ContainerStarted","Data":"6bd1fbd4fd557901cc9b28bf3a970f4643ab8e7c7f19ac36d41f8ff3ca43319d"}
Jul 17 11:02:21.834231 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:21.833948    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:21.866377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:21.866324    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb" podStartSLOduration=2.485757084 podStartE2EDuration="4.866308905s" totalImagesPullingTime="2.380551821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:02:21.865242979 +0000 UTC m=+324.722326924" watchObservedRunningTime="2026-07-17 11:02:21.866308905 +0000 UTC m=+324.723392851"
Jul 17 11:02:27.840652 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:27.840618    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-plbgb"
Jul 17 11:02:49.064820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.064787    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"]
Jul 17 11:02:49.069197 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.069178    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.072791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.072773    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 11:02:49.072791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.072781    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 11:02:49.073880 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.073865    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 11:02:49.074138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.074121    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 11:02:49.079314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.079299    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hx8rl\""
Jul 17 11:02:49.180880 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.180856    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"]
Jul 17 11:02:49.236563 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.236537    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41bfc339-8743-485e-ab59-004f8dffca27-tmp\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.236670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.236584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41bfc339-8743-485e-ab59-004f8dffca27-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.236670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.236607    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/41bfc339-8743-485e-ab59-004f8dffca27-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.236670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.236663    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/41bfc339-8743-485e-ab59-004f8dffca27-manifests\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.236810 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.236696    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjhs\" (UniqueName: \"kubernetes.io/projected/41bfc339-8743-485e-ab59-004f8dffca27-kube-api-access-hdjhs\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.337320 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.337301    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/41bfc339-8743-485e-ab59-004f8dffca27-manifests\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.337431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.337336    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjhs\" (UniqueName: \"kubernetes.io/projected/41bfc339-8743-485e-ab59-004f8dffca27-kube-api-access-hdjhs\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.337431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.337360    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41bfc339-8743-485e-ab59-004f8dffca27-tmp\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.337431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.337402    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41bfc339-8743-485e-ab59-004f8dffca27-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.337431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.337422    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/41bfc339-8743-485e-ab59-004f8dffca27-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.337691 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.337665    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/41bfc339-8743-485e-ab59-004f8dffca27-manifests\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.339690 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.339664    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41bfc339-8743-485e-ab59-004f8dffca27-tmp\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.339793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.339753    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41bfc339-8743-485e-ab59-004f8dffca27-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.339834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.339800    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/41bfc339-8743-485e-ab59-004f8dffca27-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.351803 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.351784    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjhs\" (UniqueName: \"kubernetes.io/projected/41bfc339-8743-485e-ab59-004f8dffca27-kube-api-access-hdjhs\") pod \"opendatahub-operator-controller-manager-6cd97f7fc7-b22vs\" (UID: \"41bfc339-8743-485e-ab59-004f8dffca27\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.378635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.378601    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:49.512193 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.512162    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"]
Jul 17 11:02:49.513188 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:02:49.513167    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41bfc339_8743_485e_ab59_004f8dffca27.slice/crio-dcc4611e6c3445a5bf56bbf3e967e916ea2ec3829c61a5679dc0b5bb10b44482 WatchSource:0}: Error finding container dcc4611e6c3445a5bf56bbf3e967e916ea2ec3829c61a5679dc0b5bb10b44482: Status 404 returned error can't find the container with id dcc4611e6c3445a5bf56bbf3e967e916ea2ec3829c61a5679dc0b5bb10b44482
Jul 17 11:02:49.951840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:49.951806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs" event={"ID":"41bfc339-8743-485e-ab59-004f8dffca27","Type":"ContainerStarted","Data":"dcc4611e6c3445a5bf56bbf3e967e916ea2ec3829c61a5679dc0b5bb10b44482"}
Jul 17 11:02:52.967473 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:52.967438    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs" event={"ID":"41bfc339-8743-485e-ab59-004f8dffca27","Type":"ContainerStarted","Data":"a2441f6c05440db662b6b30dda26eb5e16fe6387a10929ff1112620c4c57c2c4"}
Jul 17 11:02:53.973386 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:53.973349    2566 generic.go:356] "Generic (PLEG): container finished" podID="41bfc339-8743-485e-ab59-004f8dffca27" containerID="a2441f6c05440db662b6b30dda26eb5e16fe6387a10929ff1112620c4c57c2c4" exitCode=0
Jul 17 11:02:53.973386 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:53.973390    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs" event={"ID":"41bfc339-8743-485e-ab59-004f8dffca27","Type":"ContainerDied","Data":"a2441f6c05440db662b6b30dda26eb5e16fe6387a10929ff1112620c4c57c2c4"}
Jul 17 11:02:54.979962 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:54.979927    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs" event={"ID":"41bfc339-8743-485e-ab59-004f8dffca27","Type":"ContainerStarted","Data":"ca6a95b3ab5f2ef3b75e5ecf879fab96b73a3d58ec87d675541c7038f94f6a4d"}
Jul 17 11:02:54.980377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:54.980077    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:02:55.007320 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:02:55.007270    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs" podStartSLOduration=1.6350909059999998 podStartE2EDuration="6.00725542s" totalImagesPullingTime="3.34938366s" totalInitContainerRuntime="1.022780854s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:02:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:02:55.006207489 +0000 UTC m=+357.863291433" watchObservedRunningTime="2026-07-17 11:02:55.00725542 +0000 UTC m=+357.864339362"
Jul 17 11:03:05.987165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:05.987132    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cd97f7fc7-b22vs"
Jul 17 11:03:08.857559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.857529    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"]
Jul 17 11:03:08.866453 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.866428    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:08.871005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.870985    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 11:03:08.871667 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.871645    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 11:03:08.871771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.871689    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 11:03:08.872684 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.872665    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-24d95\""
Jul 17 11:03:08.872804 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.872684    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 11:03:08.879372 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.879352    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"]
Jul 17 11:03:08.887681 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.887662    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srm68\" (UniqueName: \"kubernetes.io/projected/693bb079-731e-47ea-b59e-fb9846c28d3d-kube-api-access-srm68\") pod \"kube-auth-proxy-c556c8c74-47nl9\" (UID: \"693bb079-731e-47ea-b59e-fb9846c28d3d\") " pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:08.887774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.887722    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/693bb079-731e-47ea-b59e-fb9846c28d3d-tmp\") pod \"kube-auth-proxy-c556c8c74-47nl9\" (UID: \"693bb079-731e-47ea-b59e-fb9846c28d3d\") " pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:08.887774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.887747    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/693bb079-731e-47ea-b59e-fb9846c28d3d-tls-certs\") pod \"kube-auth-proxy-c556c8c74-47nl9\" (UID: \"693bb079-731e-47ea-b59e-fb9846c28d3d\") " pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:08.988516 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.988483    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srm68\" (UniqueName: \"kubernetes.io/projected/693bb079-731e-47ea-b59e-fb9846c28d3d-kube-api-access-srm68\") pod \"kube-auth-proxy-c556c8c74-47nl9\" (UID: \"693bb079-731e-47ea-b59e-fb9846c28d3d\") " pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:08.988697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.988536    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/693bb079-731e-47ea-b59e-fb9846c28d3d-tmp\") pod \"kube-auth-proxy-c556c8c74-47nl9\" (UID: \"693bb079-731e-47ea-b59e-fb9846c28d3d\") " pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:08.988697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.988558    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/693bb079-731e-47ea-b59e-fb9846c28d3d-tls-certs\") pod \"kube-auth-proxy-c556c8c74-47nl9\" (UID: \"693bb079-731e-47ea-b59e-fb9846c28d3d\") " pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:08.991057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.991032    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/693bb079-731e-47ea-b59e-fb9846c28d3d-tmp\") pod \"kube-auth-proxy-c556c8c74-47nl9\" (UID: \"693bb079-731e-47ea-b59e-fb9846c28d3d\") " pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:08.991472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:08.991453    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/693bb079-731e-47ea-b59e-fb9846c28d3d-tls-certs\") pod \"kube-auth-proxy-c556c8c74-47nl9\" (UID: \"693bb079-731e-47ea-b59e-fb9846c28d3d\") " pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:09.012075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.012054    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srm68\" (UniqueName: \"kubernetes.io/projected/693bb079-731e-47ea-b59e-fb9846c28d3d-kube-api-access-srm68\") pod \"kube-auth-proxy-c556c8c74-47nl9\" (UID: \"693bb079-731e-47ea-b59e-fb9846c28d3d\") " pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:09.176623 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.176533    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"
Jul 17 11:03:09.319582 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.319556    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c556c8c74-47nl9"]
Jul 17 11:03:09.322050 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:03:09.322022    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693bb079_731e_47ea_b59e_fb9846c28d3d.slice/crio-d5ea234fa2a1feba2397983d06514d50289ca0c025f06981cf9310590973706c WatchSource:0}: Error finding container d5ea234fa2a1feba2397983d06514d50289ca0c025f06981cf9310590973706c: Status 404 returned error can't find the container with id d5ea234fa2a1feba2397983d06514d50289ca0c025f06981cf9310590973706c
Jul 17 11:03:09.842608 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.842578    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-btbtp"]
Jul 17 11:03:09.847595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.847574    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:09.870939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.870909    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-6vnlf\""
Jul 17 11:03:09.872231 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.872207    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 17 11:03:09.873665 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.873644    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-btbtp"]
Jul 17 11:03:09.897498 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.897470    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d434868f-84a2-4de3-9f5f-1f706be96803-tls-certs\") pod \"model-serving-api-54d89bd5fc-btbtp\" (UID: \"d434868f-84a2-4de3-9f5f-1f706be96803\") " pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:09.897601 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.897521    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2qh5\" (UniqueName: \"kubernetes.io/projected/d434868f-84a2-4de3-9f5f-1f706be96803-kube-api-access-g2qh5\") pod \"model-serving-api-54d89bd5fc-btbtp\" (UID: \"d434868f-84a2-4de3-9f5f-1f706be96803\") " pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:09.998078 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.998045    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2qh5\" (UniqueName: \"kubernetes.io/projected/d434868f-84a2-4de3-9f5f-1f706be96803-kube-api-access-g2qh5\") pod \"model-serving-api-54d89bd5fc-btbtp\" (UID: \"d434868f-84a2-4de3-9f5f-1f706be96803\") " pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:09.998221 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:09.998201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d434868f-84a2-4de3-9f5f-1f706be96803-tls-certs\") pod \"model-serving-api-54d89bd5fc-btbtp\" (UID: \"d434868f-84a2-4de3-9f5f-1f706be96803\") " pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:09.998324 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:03:09.998312    2566 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 17 11:03:09.998388 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:03:09.998377    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d434868f-84a2-4de3-9f5f-1f706be96803-tls-certs podName:d434868f-84a2-4de3-9f5f-1f706be96803 nodeName:}" failed. No retries permitted until 2026-07-17 11:03:10.498358667 +0000 UTC m=+373.355442590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d434868f-84a2-4de3-9f5f-1f706be96803-tls-certs") pod "model-serving-api-54d89bd5fc-btbtp" (UID: "d434868f-84a2-4de3-9f5f-1f706be96803") : secret "model-serving-api-tls" not found
Jul 17 11:03:10.026391 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:10.026356    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2qh5\" (UniqueName: \"kubernetes.io/projected/d434868f-84a2-4de3-9f5f-1f706be96803-kube-api-access-g2qh5\") pod \"model-serving-api-54d89bd5fc-btbtp\" (UID: \"d434868f-84a2-4de3-9f5f-1f706be96803\") " pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:10.050674 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:10.050641    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9" event={"ID":"693bb079-731e-47ea-b59e-fb9846c28d3d","Type":"ContainerStarted","Data":"d5ea234fa2a1feba2397983d06514d50289ca0c025f06981cf9310590973706c"}
Jul 17 11:03:10.503255 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:10.503213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d434868f-84a2-4de3-9f5f-1f706be96803-tls-certs\") pod \"model-serving-api-54d89bd5fc-btbtp\" (UID: \"d434868f-84a2-4de3-9f5f-1f706be96803\") " pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:10.506150 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:10.506120    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d434868f-84a2-4de3-9f5f-1f706be96803-tls-certs\") pod \"model-serving-api-54d89bd5fc-btbtp\" (UID: \"d434868f-84a2-4de3-9f5f-1f706be96803\") " pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:10.760006 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:10.759887    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:10.930211 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:10.930135    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-btbtp"]
Jul 17 11:03:10.933401 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:03:10.933369    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd434868f_84a2_4de3_9f5f_1f706be96803.slice/crio-10eed55574278da991e91fffeccd7b1d8ee5d9e331c3f8284830a216380a38cf WatchSource:0}: Error finding container 10eed55574278da991e91fffeccd7b1d8ee5d9e331c3f8284830a216380a38cf: Status 404 returned error can't find the container with id 10eed55574278da991e91fffeccd7b1d8ee5d9e331c3f8284830a216380a38cf
Jul 17 11:03:11.057926 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:11.057843    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-btbtp" event={"ID":"d434868f-84a2-4de3-9f5f-1f706be96803","Type":"ContainerStarted","Data":"10eed55574278da991e91fffeccd7b1d8ee5d9e331c3f8284830a216380a38cf"}
Jul 17 11:03:13.069210 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:13.069175    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9" event={"ID":"693bb079-731e-47ea-b59e-fb9846c28d3d","Type":"ContainerStarted","Data":"0c90d036a5da3aebdcba6d8b85800c227095c184c8098031af7bdfb924e94a8d"}
Jul 17 11:03:13.070726 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:13.070681    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-btbtp" event={"ID":"d434868f-84a2-4de3-9f5f-1f706be96803","Type":"ContainerStarted","Data":"5e799776f413be2a84f240ac21fabe0cf8c17d26042bf744dabca56638cfaca6"}
Jul 17 11:03:13.070843 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:13.070831    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:13.131567 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:13.131524    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-btbtp" podStartSLOduration=2.129085877 podStartE2EDuration="4.131509071s" totalImagesPullingTime="2.002423194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:03:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:03:13.131234941 +0000 UTC m=+375.988318888" watchObservedRunningTime="2026-07-17 11:03:13.131509071 +0000 UTC m=+375.988593016"
Jul 17 11:03:13.131898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:13.131873    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c556c8c74-47nl9" podStartSLOduration=2.025738781 podStartE2EDuration="5.131865684s" totalImagesPullingTime="3.106126903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:03:13.097325841 +0000 UTC m=+375.954409796" watchObservedRunningTime="2026-07-17 11:03:13.131865684 +0000 UTC m=+375.988949629"
Jul 17 11:03:14.157296 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.157266    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"]
Jul 17 11:03:14.160685 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.160668    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.163937 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.163913    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 11:03:14.164063 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.163978    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-jknhs\""
Jul 17 11:03:14.173489 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.173468    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"]
Jul 17 11:03:14.236808 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.236771    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.236981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.236843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.236981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.236905    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.236981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.236946    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.236981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.236970    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.237193 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.237002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28ec8541-42fe-4903-abb5-b71db2ecd243-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.237193 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.237112    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.237193 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.237177    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bt4r\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-kube-api-access-7bt4r\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.237323 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.237211    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.338720 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.338677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.338890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.338736    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.338890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.338771    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28ec8541-42fe-4903-abb5-b71db2ecd243-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.338890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.338803    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.338890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.338842    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bt4r\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-kube-api-access-7bt4r\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.338890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.338867    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.339176 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.338933    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.339176 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.339145    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.339255 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.339188    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.339294 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.339252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.339449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.339425    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.339618 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.339476    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.339701 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.339636    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.339782 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.339696    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28ec8541-42fe-4903-abb5-b71db2ecd243-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.341670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.341652    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.341836 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.341817    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.348368 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.348336    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.348577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.348562    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bt4r\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-kube-api-access-7bt4r\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.471471 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.471390    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:14.616457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:14.616433    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"]
Jul 17 11:03:14.618246 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:03:14.618217    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ec8541_42fe_4903_abb5_b71db2ecd243.slice/crio-6abd61855eb3832a04980f6c0731193a6033f3a5fd03051d545174b946309945 WatchSource:0}: Error finding container 6abd61855eb3832a04980f6c0731193a6033f3a5fd03051d545174b946309945: Status 404 returned error can't find the container with id 6abd61855eb3832a04980f6c0731193a6033f3a5fd03051d545174b946309945
Jul 17 11:03:15.081634 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:15.081593    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx" event={"ID":"28ec8541-42fe-4903-abb5-b71db2ecd243","Type":"ContainerStarted","Data":"6abd61855eb3832a04980f6c0731193a6033f3a5fd03051d545174b946309945"}
Jul 17 11:03:17.425396 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:17.425338    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:03:17.425674 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:17.425422    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:03:17.425674 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:17.425469    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:03:18.097758 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.097699    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx" event={"ID":"28ec8541-42fe-4903-abb5-b71db2ecd243","Type":"ContainerStarted","Data":"ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63"}
Jul 17 11:03:18.122887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.122839    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx" podStartSLOduration=1.318583382 podStartE2EDuration="4.122823243s" totalImagesPullingTime="2.804239861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:03:18.120492344 +0000 UTC m=+380.977576288" watchObservedRunningTime="2026-07-17 11:03:18.122823243 +0000 UTC m=+380.979907188"
Jul 17 11:03:18.471842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.471765    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:18.473105 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.473081    2566 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Jul 17 11:03:18.473178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.473142    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx" podUID="28ec8541-42fe-4903-abb5-b71db2ecd243" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Jul 17 11:03:18.864580 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.864552    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"]
Jul 17 11:03:18.868057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.868041    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:18.870900 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.870879    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 17 11:03:18.871016 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.870961    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 17 11:03:18.871016 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.870999    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-z79dt\""
Jul 17 11:03:18.871124 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.871112    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 17 11:03:18.885149 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.885122    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"]
Jul 17 11:03:18.977935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.977906    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:18.978083 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.977939    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b07e7bfa-cef9-4293-a26b-e12149e7042b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:18.978083 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.977984    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:18.978083 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.978028    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07e7bfa-cef9-4293-a26b-e12149e7042b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:18.978083 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.978048    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:18.978083 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.978071    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:18.978245 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:18.978098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f82dv\" (UniqueName: \"kubernetes.io/projected/b07e7bfa-cef9-4293-a26b-e12149e7042b-kube-api-access-f82dv\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.079440 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.079417    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f82dv\" (UniqueName: \"kubernetes.io/projected/b07e7bfa-cef9-4293-a26b-e12149e7042b-kube-api-access-f82dv\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.079569 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.079453    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.079569 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.079475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b07e7bfa-cef9-4293-a26b-e12149e7042b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.079569 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.079512    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.079569 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.079544    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07e7bfa-cef9-4293-a26b-e12149e7042b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.079569 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.079564    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.079826 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.079587    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.080346 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.080320    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.082081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.082053    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.082164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.082062    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.082233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.082213    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07e7bfa-cef9-4293-a26b-e12149e7042b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.082339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.082324    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b07e7bfa-cef9-4293-a26b-e12149e7042b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.089408 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.089384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07e7bfa-cef9-4293-a26b-e12149e7042b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.089594 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.089576    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f82dv\" (UniqueName: \"kubernetes.io/projected/b07e7bfa-cef9-4293-a26b-e12149e7042b-kube-api-access-f82dv\") pod \"istiod-openshift-gateway-94bb8fbfd-nntqq\" (UID: \"b07e7bfa-cef9-4293-a26b-e12149e7042b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.176921 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.176864    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:19.325820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.325777    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"]
Jul 17 11:03:19.331108 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:03:19.331078    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07e7bfa_cef9_4293_a26b_e12149e7042b.slice/crio-aaca7e6eb4ecd55162dcd07312202306641434f941872ff9a3fb3d30f3b84165 WatchSource:0}: Error finding container aaca7e6eb4ecd55162dcd07312202306641434f941872ff9a3fb3d30f3b84165: Status 404 returned error can't find the container with id aaca7e6eb4ecd55162dcd07312202306641434f941872ff9a3fb3d30f3b84165
Jul 17 11:03:19.472178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.472099    2566 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Jul 17 11:03:19.472178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:19.472164    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx" podUID="28ec8541-42fe-4903-abb5-b71db2ecd243" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Jul 17 11:03:20.109000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.108961    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq" event={"ID":"b07e7bfa-cef9-4293-a26b-e12149e7042b","Type":"ContainerStarted","Data":"aaca7e6eb4ecd55162dcd07312202306641434f941872ff9a3fb3d30f3b84165"}
Jul 17 11:03:20.472528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.472424    2566 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Jul 17 11:03:20.472981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.472774    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx" podUID="28ec8541-42fe-4903-abb5-b71db2ecd243" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Jul 17 11:03:20.474451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.474425    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"]
Jul 17 11:03:20.478024 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.478000    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"]
Jul 17 11:03:20.478129 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.478069    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:20.481308 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.481289    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:03:20.487360 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.487338    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-gsw4q\""
Jul 17 11:03:20.487602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.487584    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 17 11:03:20.488112 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.488090    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 17 11:03:20.488202 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.488140    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-v2ppg\""
Jul 17 11:03:20.493842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.493793    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"]
Jul 17 11:03:20.498892 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.498853    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"]
Jul 17 11:03:20.513846 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.513825    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"]
Jul 17 11:03:20.592747 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.592690    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2aaade70-96a8-4214-b6a7-db0ebd7bec93-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-srkz4\" (UID: \"2aaade70-96a8-4214-b6a7-db0ebd7bec93\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:03:20.592908 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.592825    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg98w\" (UniqueName: \"kubernetes.io/projected/2aaade70-96a8-4214-b6a7-db0ebd7bec93-kube-api-access-xg98w\") pod \"llmisvc-controller-manager-777c6bd7d5-srkz4\" (UID: \"2aaade70-96a8-4214-b6a7-db0ebd7bec93\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:03:20.592908 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.592882    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqtnd\" (UniqueName: \"kubernetes.io/projected/c37e45b5-fbaa-4fb9-a521-223df2aba955-kube-api-access-qqtnd\") pod \"kserve-controller-manager-d6f4b65b8-jvkbh\" (UID: \"c37e45b5-fbaa-4fb9-a521-223df2aba955\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:20.592995 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.592923    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c37e45b5-fbaa-4fb9-a521-223df2aba955-cert\") pod \"kserve-controller-manager-d6f4b65b8-jvkbh\" (UID: \"c37e45b5-fbaa-4fb9-a521-223df2aba955\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:20.694435 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.694404    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2aaade70-96a8-4214-b6a7-db0ebd7bec93-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-srkz4\" (UID: \"2aaade70-96a8-4214-b6a7-db0ebd7bec93\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:03:20.694608 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.694489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg98w\" (UniqueName: \"kubernetes.io/projected/2aaade70-96a8-4214-b6a7-db0ebd7bec93-kube-api-access-xg98w\") pod \"llmisvc-controller-manager-777c6bd7d5-srkz4\" (UID: \"2aaade70-96a8-4214-b6a7-db0ebd7bec93\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:03:20.694608 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.694532    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqtnd\" (UniqueName: \"kubernetes.io/projected/c37e45b5-fbaa-4fb9-a521-223df2aba955-kube-api-access-qqtnd\") pod \"kserve-controller-manager-d6f4b65b8-jvkbh\" (UID: \"c37e45b5-fbaa-4fb9-a521-223df2aba955\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:20.694608 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.694567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c37e45b5-fbaa-4fb9-a521-223df2aba955-cert\") pod \"kserve-controller-manager-d6f4b65b8-jvkbh\" (UID: \"c37e45b5-fbaa-4fb9-a521-223df2aba955\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:20.697075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.697053    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c37e45b5-fbaa-4fb9-a521-223df2aba955-cert\") pod \"kserve-controller-manager-d6f4b65b8-jvkbh\" (UID: \"c37e45b5-fbaa-4fb9-a521-223df2aba955\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:20.697075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.697072    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2aaade70-96a8-4214-b6a7-db0ebd7bec93-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-srkz4\" (UID: \"2aaade70-96a8-4214-b6a7-db0ebd7bec93\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:03:20.707044 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.707015    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg98w\" (UniqueName: \"kubernetes.io/projected/2aaade70-96a8-4214-b6a7-db0ebd7bec93-kube-api-access-xg98w\") pod \"llmisvc-controller-manager-777c6bd7d5-srkz4\" (UID: \"2aaade70-96a8-4214-b6a7-db0ebd7bec93\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:03:20.715043 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.715022    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqtnd\" (UniqueName: \"kubernetes.io/projected/c37e45b5-fbaa-4fb9-a521-223df2aba955-kube-api-access-qqtnd\") pod \"kserve-controller-manager-d6f4b65b8-jvkbh\" (UID: \"c37e45b5-fbaa-4fb9-a521-223df2aba955\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:20.795262 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.795170    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:20.801678 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:20.801645    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:03:21.113274 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:21.113220    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx" podUID="28ec8541-42fe-4903-abb5-b71db2ecd243" containerName="istio-proxy" containerID="cri-o://ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63" gracePeriod=30
Jul 17 11:03:21.457379 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:21.457304    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:03:21.458063 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:21.458023    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:03:21.553965 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:21.553935    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"]
Jul 17 11:03:21.556718 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:03:21.556676    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2aaade70_96a8_4214_b6a7_db0ebd7bec93.slice/crio-cf7060d3b0b2b98027cf15ac1178e9b466e4f33b2d9576be9b60e09b53d99cc6 WatchSource:0}: Error finding container cf7060d3b0b2b98027cf15ac1178e9b466e4f33b2d9576be9b60e09b53d99cc6: Status 404 returned error can't find the container with id cf7060d3b0b2b98027cf15ac1178e9b466e4f33b2d9576be9b60e09b53d99cc6
Jul 17 11:03:21.567787 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:21.567765    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"]
Jul 17 11:03:21.569922 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:03:21.569892    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc37e45b5_fbaa_4fb9_a521_223df2aba955.slice/crio-24266ce3f6b502495f6fc9ffe6b49c92dca179ec67bc96e79e0ffa379cf8456a WatchSource:0}: Error finding container 24266ce3f6b502495f6fc9ffe6b49c92dca179ec67bc96e79e0ffa379cf8456a: Status 404 returned error can't find the container with id 24266ce3f6b502495f6fc9ffe6b49c92dca179ec67bc96e79e0ffa379cf8456a
Jul 17 11:03:22.120046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:22.119927    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq" event={"ID":"b07e7bfa-cef9-4293-a26b-e12149e7042b","Type":"ContainerStarted","Data":"f52ba8aa7c2dadb870f5a4655e1e7f62a260e4ebb098705d581d597250571b6e"}
Jul 17 11:03:22.120385 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:22.120111    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:22.121672 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:22.121644    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh" event={"ID":"c37e45b5-fbaa-4fb9-a521-223df2aba955","Type":"ContainerStarted","Data":"24266ce3f6b502495f6fc9ffe6b49c92dca179ec67bc96e79e0ffa379cf8456a"}
Jul 17 11:03:22.121813 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:22.121755    2566 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-nntqq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 11:03:22.121879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:22.121813    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq" podUID="b07e7bfa-cef9-4293-a26b-e12149e7042b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 11:03:22.122791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:22.122766    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4" event={"ID":"2aaade70-96a8-4214-b6a7-db0ebd7bec93","Type":"ContainerStarted","Data":"cf7060d3b0b2b98027cf15ac1178e9b466e4f33b2d9576be9b60e09b53d99cc6"}
Jul 17 11:03:23.130193 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:23.130161    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq"
Jul 17 11:03:23.195828 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:23.195537    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nntqq" podStartSLOduration=3.072526204 podStartE2EDuration="5.195513551s" totalImagesPullingTime="2.122987347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:03:22.164201628 +0000 UTC m=+385.021285573" watchObservedRunningTime="2026-07-17 11:03:23.195513551 +0000 UTC m=+386.052597497"
Jul 17 11:03:24.080565 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:24.080535    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-btbtp"
Jul 17 11:03:25.140398 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:25.140363    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh" event={"ID":"c37e45b5-fbaa-4fb9-a521-223df2aba955","Type":"ContainerStarted","Data":"54ab648b97256230ce31c32369c06497f7326867686517c7dc4cbb57c7faa833"}
Jul 17 11:03:25.140851 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:25.140458    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:25.159959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:25.159911    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh" podStartSLOduration=2.546862425 podStartE2EDuration="5.15989194s" totalImagesPullingTime="2.613029515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:03:25.15848708 +0000 UTC m=+388.015571017" watchObservedRunningTime="2026-07-17 11:03:25.15989194 +0000 UTC m=+388.016975887"
Jul 17 11:03:26.147944 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.147915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4" event={"ID":"2aaade70-96a8-4214-b6a7-db0ebd7bec93","Type":"ContainerStarted","Data":"3b1dc381a89e2cf03937e85e6e60ff6a6c20d6faf3fd62dd3e24edcf10530d0c"}
Jul 17 11:03:26.148333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.148079    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:03:26.179591 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.179534    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4" podStartSLOduration=2.588646962 podStartE2EDuration="6.179513852s" totalImagesPullingTime="3.59086689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:03:26.178202597 +0000 UTC m=+389.035286555" watchObservedRunningTime="2026-07-17 11:03:26.179513852 +0000 UTC m=+389.036597797"
Jul 17 11:03:26.374069 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.374016    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:26.452584 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452555    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-socket\") pod \"28ec8541-42fe-4903-abb5-b71db2ecd243\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") "
Jul 17 11:03:26.452738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452615    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-token\") pod \"28ec8541-42fe-4903-abb5-b71db2ecd243\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") "
Jul 17 11:03:26.452738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452661    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28ec8541-42fe-4903-abb5-b71db2ecd243-istiod-ca-cert\") pod \"28ec8541-42fe-4903-abb5-b71db2ecd243\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") "
Jul 17 11:03:26.452738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452679    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-envoy\") pod \"28ec8541-42fe-4903-abb5-b71db2ecd243\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") "
Jul 17 11:03:26.452738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452688    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "28ec8541-42fe-4903-abb5-b71db2ecd243" (UID: "28ec8541-42fe-4903-abb5-b71db2ecd243"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:03:26.452738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452737    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-certs\") pod \"28ec8541-42fe-4903-abb5-b71db2ecd243\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") "
Jul 17 11:03:26.452953 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452765    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-data\") pod \"28ec8541-42fe-4903-abb5-b71db2ecd243\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") "
Jul 17 11:03:26.452953 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452783    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-podinfo\") pod \"28ec8541-42fe-4903-abb5-b71db2ecd243\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") "
Jul 17 11:03:26.452953 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452823    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bt4r\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-kube-api-access-7bt4r\") pod \"28ec8541-42fe-4903-abb5-b71db2ecd243\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") "
Jul 17 11:03:26.452953 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.452907    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-credential-socket\") pod \"28ec8541-42fe-4903-abb5-b71db2ecd243\" (UID: \"28ec8541-42fe-4903-abb5-b71db2ecd243\") "
Jul 17 11:03:26.453169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.453009    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-data" (OuterVolumeSpecName: "istio-data") pod "28ec8541-42fe-4903-abb5-b71db2ecd243" (UID: "28ec8541-42fe-4903-abb5-b71db2ecd243"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:03:26.453169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.453026    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "28ec8541-42fe-4903-abb5-b71db2ecd243" (UID: "28ec8541-42fe-4903-abb5-b71db2ecd243"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:03:26.453169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.453041    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ec8541-42fe-4903-abb5-b71db2ecd243-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "28ec8541-42fe-4903-abb5-b71db2ecd243" (UID: "28ec8541-42fe-4903-abb5-b71db2ecd243"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:03:26.453319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.453208    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:03:26.453319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.453221    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:03:26.453319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.453230    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:03:26.453319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.453241    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28ec8541-42fe-4903-abb5-b71db2ecd243-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:03:26.453319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.453224    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "28ec8541-42fe-4903-abb5-b71db2ecd243" (UID: "28ec8541-42fe-4903-abb5-b71db2ecd243"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:03:26.454996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.454968    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "28ec8541-42fe-4903-abb5-b71db2ecd243" (UID: "28ec8541-42fe-4903-abb5-b71db2ecd243"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:03:26.455121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.455000    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-token" (OuterVolumeSpecName: "istio-token") pod "28ec8541-42fe-4903-abb5-b71db2ecd243" (UID: "28ec8541-42fe-4903-abb5-b71db2ecd243"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:03:26.455349 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.455335    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "28ec8541-42fe-4903-abb5-b71db2ecd243" (UID: "28ec8541-42fe-4903-abb5-b71db2ecd243"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:03:26.455394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.455365    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-kube-api-access-7bt4r" (OuterVolumeSpecName: "kube-api-access-7bt4r") pod "28ec8541-42fe-4903-abb5-b71db2ecd243" (UID: "28ec8541-42fe-4903-abb5-b71db2ecd243"). InnerVolumeSpecName "kube-api-access-7bt4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:03:26.553990 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.553963    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:03:26.553990 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.553985    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:03:26.553990 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.553994    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bt4r\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-kube-api-access-7bt4r\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:03:26.554170 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.554004    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28ec8541-42fe-4903-abb5-b71db2ecd243-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:03:26.554170 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:26.554014    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28ec8541-42fe-4903-abb5-b71db2ecd243-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:03:27.153378 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.153342    2566 generic.go:356] "Generic (PLEG): container finished" podID="28ec8541-42fe-4903-abb5-b71db2ecd243" containerID="ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63" exitCode=0
Jul 17 11:03:27.153820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.153387    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"
Jul 17 11:03:27.153820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.153430    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx" event={"ID":"28ec8541-42fe-4903-abb5-b71db2ecd243","Type":"ContainerDied","Data":"ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63"}
Jul 17 11:03:27.153820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.153478    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx" event={"ID":"28ec8541-42fe-4903-abb5-b71db2ecd243","Type":"ContainerDied","Data":"6abd61855eb3832a04980f6c0731193a6033f3a5fd03051d545174b946309945"}
Jul 17 11:03:27.153820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.153498    2566 scope.go:117] "RemoveContainer" containerID="ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63"
Jul 17 11:03:27.169576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.169558    2566 scope.go:117] "RemoveContainer" containerID="ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63"
Jul 17 11:03:27.169840 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:03:27.169821    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63\": container with ID starting with ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63 not found: ID does not exist" containerID="ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63"
Jul 17 11:03:27.169897 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.169849    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63"} err="failed to get container status \"ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63\": rpc error: code = NotFound desc = could not find container \"ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63\": container with ID starting with ec8892c84d34ae1b9abb671bec1668237930e8790aa75771f8d982ed26eafa63 not found: ID does not exist"
Jul 17 11:03:27.182492 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.182471    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"]
Jul 17 11:03:27.187480 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.187458    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-fgbwx"]
Jul 17 11:03:27.657917 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:27.657889    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ec8541-42fe-4903-abb5-b71db2ecd243" path="/var/lib/kubelet/pods/28ec8541-42fe-4903-abb5-b71db2ecd243/volumes"
Jul 17 11:03:56.152674 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:56.152641    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkbh"
Jul 17 11:03:57.155729 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:03:57.155683    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-srkz4"
Jul 17 11:04:48.416397 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.416359    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"]
Jul 17 11:04:48.416837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.416692    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ec8541-42fe-4903-abb5-b71db2ecd243" containerName="istio-proxy"
Jul 17 11:04:48.416837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.416719    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ec8541-42fe-4903-abb5-b71db2ecd243" containerName="istio-proxy"
Jul 17 11:04:48.416837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.416795    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ec8541-42fe-4903-abb5-b71db2ecd243" containerName="istio-proxy"
Jul 17 11:04:48.418884 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.418862    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:04:48.422090 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.422066    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 11:04:48.422253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.422089    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-87w4x\""
Jul 17 11:04:48.422253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.422123    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 11:04:48.432145 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.432126    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"]
Jul 17 11:04:48.519737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.519693    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqjnf\" (UniqueName: \"kubernetes.io/projected/8cd09cba-b265-40d7-9e9f-6da91f60e099-kube-api-access-sqjnf\") pod \"limitador-operator-controller-manager-859d7b78bd-8rdt8\" (UID: \"8cd09cba-b265-40d7-9e9f-6da91f60e099\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:04:48.621066 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.621034    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqjnf\" (UniqueName: \"kubernetes.io/projected/8cd09cba-b265-40d7-9e9f-6da91f60e099-kube-api-access-sqjnf\") pod \"limitador-operator-controller-manager-859d7b78bd-8rdt8\" (UID: \"8cd09cba-b265-40d7-9e9f-6da91f60e099\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:04:48.631011 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.630985    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqjnf\" (UniqueName: \"kubernetes.io/projected/8cd09cba-b265-40d7-9e9f-6da91f60e099-kube-api-access-sqjnf\") pod \"limitador-operator-controller-manager-859d7b78bd-8rdt8\" (UID: \"8cd09cba-b265-40d7-9e9f-6da91f60e099\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:04:48.728923 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.728847    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:04:48.860600 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:48.860574    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"]
Jul 17 11:04:48.863296 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:04:48.863271    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd09cba_b265_40d7_9e9f_6da91f60e099.slice/crio-cca9ea0bd8e8192ff51f71c0b0e0b6b45bd3844d2673cec13079d4368b2cc1d1 WatchSource:0}: Error finding container cca9ea0bd8e8192ff51f71c0b0e0b6b45bd3844d2673cec13079d4368b2cc1d1: Status 404 returned error can't find the container with id cca9ea0bd8e8192ff51f71c0b0e0b6b45bd3844d2673cec13079d4368b2cc1d1
Jul 17 11:04:49.516333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:49.516299    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8" event={"ID":"8cd09cba-b265-40d7-9e9f-6da91f60e099","Type":"ContainerStarted","Data":"cca9ea0bd8e8192ff51f71c0b0e0b6b45bd3844d2673cec13079d4368b2cc1d1"}
Jul 17 11:04:51.527496 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:51.527407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8" event={"ID":"8cd09cba-b265-40d7-9e9f-6da91f60e099","Type":"ContainerStarted","Data":"68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded"}
Jul 17 11:04:51.527496 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:51.527468    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:04:51.549794 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:51.549745    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8" podStartSLOduration=1.289897598 podStartE2EDuration="3.549729752s" totalImagesPullingTime="2.259832154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:04:51.548495629 +0000 UTC m=+474.405579573" watchObservedRunningTime="2026-07-17 11:04:51.549729752 +0000 UTC m=+474.406813698"
Jul 17 11:04:53.970233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:53.970193    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6587cf4f48-wj762"]
Jul 17 11:04:53.973846 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:53.973818    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:53.988018 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:53.987992    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6587cf4f48-wj762"]
Jul 17 11:04:54.068850 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.068813    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-console-serving-cert\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.069015 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.068868    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-console-oauth-config\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.069015 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.068895    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9cpf\" (UniqueName: \"kubernetes.io/projected/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-kube-api-access-x9cpf\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.069015 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.068953    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-console-config\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.069164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.069046    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-service-ca\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.069164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.069067    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-oauth-serving-cert\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.069164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.069098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-trusted-ca-bundle\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.169669 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.169635    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-service-ca\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.169669 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.169669    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-oauth-serving-cert\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.169945 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.169700    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-trusted-ca-bundle\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.170035 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.170011    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-console-serving-cert\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.170105 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.170089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-console-oauth-config\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.170175 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.170121    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9cpf\" (UniqueName: \"kubernetes.io/projected/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-kube-api-access-x9cpf\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.170175 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.170152    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-console-config\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.170565 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.170537    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-service-ca\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.170686 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.170575    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-trusted-ca-bundle\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.170786 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.170727    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-oauth-serving-cert\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.170878 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.170852    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-console-config\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.172565 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.172548    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-console-oauth-config\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.172638 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.172608    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-console-serving-cert\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.185808 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.185788    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9cpf\" (UniqueName: \"kubernetes.io/projected/3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a-kube-api-access-x9cpf\") pod \"console-6587cf4f48-wj762\" (UID: \"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a\") " pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.283575 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.283489    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:04:54.414779 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.414754    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6587cf4f48-wj762"]
Jul 17 11:04:54.416360 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:04:54.416337    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2d8d4e_31b8_49cd_81b9_fd3a6c5dc78a.slice/crio-c545fa363484e8d52d030e620fff366a6ab50f59c79b8ff9e03f8187778b80ed WatchSource:0}: Error finding container c545fa363484e8d52d030e620fff366a6ab50f59c79b8ff9e03f8187778b80ed: Status 404 returned error can't find the container with id c545fa363484e8d52d030e620fff366a6ab50f59c79b8ff9e03f8187778b80ed
Jul 17 11:04:54.547060 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.546972    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6587cf4f48-wj762" event={"ID":"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a","Type":"ContainerStarted","Data":"b1af3e2ca92c23b4aa784666e86b638cffe6f013a4c1249cae5463789a37971a"}
Jul 17 11:04:54.547060 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.547020    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6587cf4f48-wj762" event={"ID":"3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a","Type":"ContainerStarted","Data":"c545fa363484e8d52d030e620fff366a6ab50f59c79b8ff9e03f8187778b80ed"}
Jul 17 11:04:54.566986 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:04:54.566942    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6587cf4f48-wj762" podStartSLOduration=1.56692665 podStartE2EDuration="1.56692665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:04:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:04:54.565801492 +0000 UTC m=+477.422885449" watchObservedRunningTime="2026-07-17 11:04:54.56692665 +0000 UTC m=+477.424010595"
Jul 17 11:05:02.539304 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:02.539267    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:05:04.284080 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:04.284043    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:05:04.284528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:04.284109    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:05:04.288645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:04.288624    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:05:04.594009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:04.593982    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6587cf4f48-wj762"
Jul 17 11:05:04.692501 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:04.692470    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7459678d7d-x675z"]
Jul 17 11:05:06.675599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.675564    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"]
Jul 17 11:05:06.676042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.675789    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" containerName="manager" containerID="cri-o://68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded" gracePeriod=2
Jul 17 11:05:06.692511 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.692410    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"]
Jul 17 11:05:06.695754 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:05:06.695724    2566 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8cd09cba-b265-40d7-9e9f-6da91f60e099\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-8rdt8\": pods \"limitador-operator-controller-manager-859d7b78bd-8rdt8\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:05:06.709630 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.709606    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"]
Jul 17 11:05:06.710087 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.710069    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" containerName="manager"
Jul 17 11:05:06.710185 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.710089    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" containerName="manager"
Jul 17 11:05:06.710185 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.710176    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" containerName="manager"
Jul 17 11:05:06.713469 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.713449    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"
Jul 17 11:05:06.715954 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:05:06.715929    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8rdt8\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:05:06.734107 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.734084    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"]
Jul 17 11:05:06.777051 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.777029    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chclz\" (UniqueName: \"kubernetes.io/projected/d3784d55-7352-4002-a85a-d40559e24d64-kube-api-access-chclz\") pod \"limitador-operator-controller-manager-859d7b78bd-d4knq\" (UID: \"d3784d55-7352-4002-a85a-d40559e24d64\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"
Jul 17 11:05:06.878491 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.878461    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chclz\" (UniqueName: \"kubernetes.io/projected/d3784d55-7352-4002-a85a-d40559e24d64-kube-api-access-chclz\") pod \"limitador-operator-controller-manager-859d7b78bd-d4knq\" (UID: \"d3784d55-7352-4002-a85a-d40559e24d64\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"
Jul 17 11:05:06.897016 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.896994    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chclz\" (UniqueName: \"kubernetes.io/projected/d3784d55-7352-4002-a85a-d40559e24d64-kube-api-access-chclz\") pod \"limitador-operator-controller-manager-859d7b78bd-d4knq\" (UID: \"d3784d55-7352-4002-a85a-d40559e24d64\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"
Jul 17 11:05:06.912223 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.912202    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:05:06.914655 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:05:06.914627    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8rdt8\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:05:06.979559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.979495    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqjnf\" (UniqueName: \"kubernetes.io/projected/8cd09cba-b265-40d7-9e9f-6da91f60e099-kube-api-access-sqjnf\") pod \"8cd09cba-b265-40d7-9e9f-6da91f60e099\" (UID: \"8cd09cba-b265-40d7-9e9f-6da91f60e099\") "
Jul 17 11:05:06.981557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:06.981537    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cd09cba-b265-40d7-9e9f-6da91f60e099-kube-api-access-sqjnf" (OuterVolumeSpecName: "kube-api-access-sqjnf") pod "8cd09cba-b265-40d7-9e9f-6da91f60e099" (UID: "8cd09cba-b265-40d7-9e9f-6da91f60e099"). InnerVolumeSpecName "kube-api-access-sqjnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:05:07.073813 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.073783    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"
Jul 17 11:05:07.080794 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.080771    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqjnf\" (UniqueName: \"kubernetes.io/projected/8cd09cba-b265-40d7-9e9f-6da91f60e099-kube-api-access-sqjnf\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:05:07.215410 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.215357    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"]
Jul 17 11:05:07.217734 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:05:07.217680    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3784d55_7352_4002_a85a_d40559e24d64.slice/crio-f13479932c84bdf02361d39f36e1ffda52a4a8e4ec3dcca54db9a279a3ca7712 WatchSource:0}: Error finding container f13479932c84bdf02361d39f36e1ffda52a4a8e4ec3dcca54db9a279a3ca7712: Status 404 returned error can't find the container with id f13479932c84bdf02361d39f36e1ffda52a4a8e4ec3dcca54db9a279a3ca7712
Jul 17 11:05:07.611147 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.611107    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq" event={"ID":"d3784d55-7352-4002-a85a-d40559e24d64","Type":"ContainerStarted","Data":"350caa6db690c25b0d68f13c51b54aad871620346528697c46bac0a2735ce6ea"}
Jul 17 11:05:07.611354 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.611309    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"
Jul 17 11:05:07.611426 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.611356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq" event={"ID":"d3784d55-7352-4002-a85a-d40559e24d64","Type":"ContainerStarted","Data":"f13479932c84bdf02361d39f36e1ffda52a4a8e4ec3dcca54db9a279a3ca7712"}
Jul 17 11:05:07.612312 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.612289    2566 generic.go:356] "Generic (PLEG): container finished" podID="8cd09cba-b265-40d7-9e9f-6da91f60e099" containerID="68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded" exitCode=0
Jul 17 11:05:07.612426 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.612329    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:05:07.612426 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.612369    2566 scope.go:117] "RemoveContainer" containerID="68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded"
Jul 17 11:05:07.613583 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:05:07.613557    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8rdt8\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:05:07.621241 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.621225    2566 scope.go:117] "RemoveContainer" containerID="68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded"
Jul 17 11:05:07.621466 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:05:07.621449    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded\": container with ID starting with 68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded not found: ID does not exist" containerID="68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded"
Jul 17 11:05:07.621510 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.621480    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded"} err="failed to get container status \"68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded\": rpc error: code = NotFound desc = could not find container \"68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded\": container with ID starting with 68de4a2abb93ba096884be5f269e78110f9b9b3c35daa37e97195bc48ff58ded not found: ID does not exist"
Jul 17 11:05:07.630997 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.630942    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq" podStartSLOduration=1.6309280940000002 podStartE2EDuration="1.630928094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:05:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:05:07.630403457 +0000 UTC m=+490.487487427" watchObservedRunningTime="2026-07-17 11:05:07.630928094 +0000 UTC m=+490.488012039"
Jul 17 11:05:07.632403 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:05:07.632376    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8rdt8\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:05:07.656901 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:05:07.656871    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8rdt8\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8rdt8"
Jul 17 11:05:07.658029 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:07.658012    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cd09cba-b265-40d7-9e9f-6da91f60e099" path="/var/lib/kubelet/pods/8cd09cba-b265-40d7-9e9f-6da91f60e099/volumes"
Jul 17 11:05:18.619848 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:18.619815    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d4knq"
Jul 17 11:05:29.716880 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:29.716843    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7459678d7d-x675z" podUID="bfd1be43-7c10-4b27-8de0-aa9e1834dd61" containerName="console" containerID="cri-o://d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d" gracePeriod=15
Jul 17 11:05:29.966976 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:29.966925    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:05:30.065873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.065843    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjjwd\" (UniqueName: \"kubernetes.io/projected/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-kube-api-access-jjjwd\") pod \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") "
Jul 17 11:05:30.066013 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.065894    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-serving-cert\") pod \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") "
Jul 17 11:05:30.066013 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.065941    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-trusted-ca-bundle\") pod \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") "
Jul 17 11:05:30.066013 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.065963    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-oauth-serving-cert\") pod \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") "
Jul 17 11:05:30.066013 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.065999    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-service-ca\") pod \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") "
Jul 17 11:05:30.066229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.066031    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-oauth-config\") pod \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") "
Jul 17 11:05:30.066229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.066080    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-config\") pod \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\" (UID: \"bfd1be43-7c10-4b27-8de0-aa9e1834dd61\") "
Jul 17 11:05:30.066415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.066385    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bfd1be43-7c10-4b27-8de0-aa9e1834dd61" (UID: "bfd1be43-7c10-4b27-8de0-aa9e1834dd61"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:05:30.066415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.066401    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bfd1be43-7c10-4b27-8de0-aa9e1834dd61" (UID: "bfd1be43-7c10-4b27-8de0-aa9e1834dd61"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:05:30.066583 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.066546    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-service-ca" (OuterVolumeSpecName: "service-ca") pod "bfd1be43-7c10-4b27-8de0-aa9e1834dd61" (UID: "bfd1be43-7c10-4b27-8de0-aa9e1834dd61"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:05:30.066719 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.066598    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-config" (OuterVolumeSpecName: "console-config") pod "bfd1be43-7c10-4b27-8de0-aa9e1834dd61" (UID: "bfd1be43-7c10-4b27-8de0-aa9e1834dd61"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:05:30.068087 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.068057    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-kube-api-access-jjjwd" (OuterVolumeSpecName: "kube-api-access-jjjwd") pod "bfd1be43-7c10-4b27-8de0-aa9e1834dd61" (UID: "bfd1be43-7c10-4b27-8de0-aa9e1834dd61"). InnerVolumeSpecName "kube-api-access-jjjwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:05:30.068224 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.068199    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bfd1be43-7c10-4b27-8de0-aa9e1834dd61" (UID: "bfd1be43-7c10-4b27-8de0-aa9e1834dd61"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:05:30.068224 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.068213    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bfd1be43-7c10-4b27-8de0-aa9e1834dd61" (UID: "bfd1be43-7c10-4b27-8de0-aa9e1834dd61"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:05:30.167474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.167438    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjjwd\" (UniqueName: \"kubernetes.io/projected/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-kube-api-access-jjjwd\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:05:30.167635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.167486    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-serving-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:05:30.167635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.167496    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-trusted-ca-bundle\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:05:30.167635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.167506    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-oauth-serving-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:05:30.167635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.167514    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:05:30.167635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.167524    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-oauth-config\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:05:30.167635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.167532    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfd1be43-7c10-4b27-8de0-aa9e1834dd61-console-config\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:05:30.715581 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.715546    2566 generic.go:356] "Generic (PLEG): container finished" podID="bfd1be43-7c10-4b27-8de0-aa9e1834dd61" containerID="d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d" exitCode=0
Jul 17 11:05:30.715769 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.715589    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7459678d7d-x675z"
Jul 17 11:05:30.715769 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.715623    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7459678d7d-x675z" event={"ID":"bfd1be43-7c10-4b27-8de0-aa9e1834dd61","Type":"ContainerDied","Data":"d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d"}
Jul 17 11:05:30.715769 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.715661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7459678d7d-x675z" event={"ID":"bfd1be43-7c10-4b27-8de0-aa9e1834dd61","Type":"ContainerDied","Data":"83f0cb3f626bb47f68a9dc0bf478736c5a0c77c5d9c28154fd216391b04a234e"}
Jul 17 11:05:30.715769 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.715685    2566 scope.go:117] "RemoveContainer" containerID="d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d"
Jul 17 11:05:30.725445 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.725267    2566 scope.go:117] "RemoveContainer" containerID="d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d"
Jul 17 11:05:30.725652 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:05:30.725494    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d\": container with ID starting with d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d not found: ID does not exist" containerID="d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d"
Jul 17 11:05:30.725652 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.725518    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d"} err="failed to get container status \"d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d\": rpc error: code = NotFound desc = could not find container \"d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d\": container with ID starting with d51058908a0b74342ec26be7c5051b09fdb228e8411c932de1f8b3f276640d5d not found: ID does not exist"
Jul 17 11:05:30.739971 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.739945    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7459678d7d-x675z"]
Jul 17 11:05:30.746682 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:30.746663    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7459678d7d-x675z"]
Jul 17 11:05:31.658008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:31.657979    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfd1be43-7c10-4b27-8de0-aa9e1834dd61" path="/var/lib/kubelet/pods/bfd1be43-7c10-4b27-8de0-aa9e1834dd61/volumes"
Jul 17 11:05:48.347715 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.347668    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"]
Jul 17 11:05:48.348102 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.348043    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd1be43-7c10-4b27-8de0-aa9e1834dd61" containerName="console"
Jul 17 11:05:48.348102 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.348056    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd1be43-7c10-4b27-8de0-aa9e1834dd61" containerName="console"
Jul 17 11:05:48.348171 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.348114    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfd1be43-7c10-4b27-8de0-aa9e1834dd61" containerName="console"
Jul 17 11:05:48.352546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.352508    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.355881 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.355857    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zgfw2\""
Jul 17 11:05:48.366782 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.366749    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"]
Jul 17 11:05:48.521128 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.521085    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.521128 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.521139    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.521348 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.521197    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.521348 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.521232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.521348 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.521281    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftssj\" (UniqueName: \"kubernetes.io/projected/93486233-9ab9-4859-ba3e-711e53d4c05a-kube-api-access-ftssj\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.521446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.521355    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.521446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.521380    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93486233-9ab9-4859-ba3e-711e53d4c05a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.521446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.521400    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.521446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.521417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.622627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.622541    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftssj\" (UniqueName: \"kubernetes.io/projected/93486233-9ab9-4859-ba3e-711e53d4c05a-kube-api-access-ftssj\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.622627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.622581    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.622627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.622601    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93486233-9ab9-4859-ba3e-711e53d4c05a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.622877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.622735    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.622877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.622766    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.622877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.622839    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.622877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.622861    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.623042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.622902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.623042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.622935    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.623125 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.623079    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.623206 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.623185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.623657 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.623581    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.625333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.623853    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93486233-9ab9-4859-ba3e-711e53d4c05a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.625333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.623970    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.625994 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.625972    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.630737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.630670    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.633033 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.633015    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93486233-9ab9-4859-ba3e-711e53d4c05a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.633165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.633148    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftssj\" (UniqueName: \"kubernetes.io/projected/93486233-9ab9-4859-ba3e-711e53d4c05a-kube-api-access-ftssj\") pod \"maas-default-gateway-openshift-default-56654dcf7-hkwn8\" (UID: \"93486233-9ab9-4859-ba3e-711e53d4c05a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.661628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.661599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:48.800420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.800398    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"]
Jul 17 11:05:48.802881 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:05:48.802854    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93486233_9ab9_4859_ba3e_711e53d4c05a.slice/crio-206f2da2b0b5faea7d7784fa365e7a9d492b3cc3b15fdf125084a3f4d8ff0044 WatchSource:0}: Error finding container 206f2da2b0b5faea7d7784fa365e7a9d492b3cc3b15fdf125084a3f4d8ff0044: Status 404 returned error can't find the container with id 206f2da2b0b5faea7d7784fa365e7a9d492b3cc3b15fdf125084a3f4d8ff0044
Jul 17 11:05:48.804783 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.804740    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:05:48.804886 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.804823    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:05:48.804886 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:48.804874    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:05:49.801951 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:49.801912    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8" event={"ID":"93486233-9ab9-4859-ba3e-711e53d4c05a","Type":"ContainerStarted","Data":"d6abe77a5adf1d426da5bf41d4e35f4adca5d46330c5b4cca741361ef4c3b91c"}
Jul 17 11:05:49.802537 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:49.801970    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8" event={"ID":"93486233-9ab9-4859-ba3e-711e53d4c05a","Type":"ContainerStarted","Data":"206f2da2b0b5faea7d7784fa365e7a9d492b3cc3b15fdf125084a3f4d8ff0044"}
Jul 17 11:05:49.827389 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:49.827342    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8" podStartSLOduration=1.827327033 podStartE2EDuration="1.827327033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:05:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:05:49.826057449 +0000 UTC m=+532.683141404" watchObservedRunningTime="2026-07-17 11:05:49.827327033 +0000 UTC m=+532.684411008"
Jul 17 11:05:50.662019 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:50.661935    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:50.666844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:50.666814    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:50.806780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:50.806753    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:05:50.807548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:05:50.807528    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:06:05.374809 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.374770    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:06:05.379574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.379552    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:05.382238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.382218    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5xfhp\""
Jul 17 11:06:05.382324 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.382218    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 17 11:06:05.389217 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.389194    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:06:05.427071 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.427042    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:06:05.461876 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.461839    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snk57\" (UniqueName: \"kubernetes.io/projected/00908a0f-f8d4-42c7-a1c6-6014d807b6f9-kube-api-access-snk57\") pod \"limitador-limitador-54d6d988d5-nclqh\" (UID: \"00908a0f-f8d4-42c7-a1c6-6014d807b6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:05.461996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.461893    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00908a0f-f8d4-42c7-a1c6-6014d807b6f9-config-file\") pod \"limitador-limitador-54d6d988d5-nclqh\" (UID: \"00908a0f-f8d4-42c7-a1c6-6014d807b6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:05.562524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.562494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snk57\" (UniqueName: \"kubernetes.io/projected/00908a0f-f8d4-42c7-a1c6-6014d807b6f9-kube-api-access-snk57\") pod \"limitador-limitador-54d6d988d5-nclqh\" (UID: \"00908a0f-f8d4-42c7-a1c6-6014d807b6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:05.562693 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.562555    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00908a0f-f8d4-42c7-a1c6-6014d807b6f9-config-file\") pod \"limitador-limitador-54d6d988d5-nclqh\" (UID: \"00908a0f-f8d4-42c7-a1c6-6014d807b6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:05.563142 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.563123    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00908a0f-f8d4-42c7-a1c6-6014d807b6f9-config-file\") pod \"limitador-limitador-54d6d988d5-nclqh\" (UID: \"00908a0f-f8d4-42c7-a1c6-6014d807b6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:05.572980 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.572957    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snk57\" (UniqueName: \"kubernetes.io/projected/00908a0f-f8d4-42c7-a1c6-6014d807b6f9-kube-api-access-snk57\") pod \"limitador-limitador-54d6d988d5-nclqh\" (UID: \"00908a0f-f8d4-42c7-a1c6-6014d807b6f9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:05.690316 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.690246    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:05.823677 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.823657    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:06:05.825979 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:06:05.825947    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00908a0f_f8d4_42c7_a1c6_6014d807b6f9.slice/crio-d94693884ad3e9150e1f707ce69c8887ec95b71347c1b2f7aa96b06e452bfa5f WatchSource:0}: Error finding container d94693884ad3e9150e1f707ce69c8887ec95b71347c1b2f7aa96b06e452bfa5f: Status 404 returned error can't find the container with id d94693884ad3e9150e1f707ce69c8887ec95b71347c1b2f7aa96b06e452bfa5f
Jul 17 11:06:05.873986 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:05.873956    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh" event={"ID":"00908a0f-f8d4-42c7-a1c6-6014d807b6f9","Type":"ContainerStarted","Data":"d94693884ad3e9150e1f707ce69c8887ec95b71347c1b2f7aa96b06e452bfa5f"}
Jul 17 11:06:08.889767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:08.889737    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh" event={"ID":"00908a0f-f8d4-42c7-a1c6-6014d807b6f9","Type":"ContainerStarted","Data":"c65507a6cc7c404f3e80fcac8b0d3295d53d1d0d2aba95beed51d673880b5f5a"}
Jul 17 11:06:08.890140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:08.889866    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:08.930178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:08.930127    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh" podStartSLOduration=1.543711741 podStartE2EDuration="3.930112953s" totalImagesPullingTime="2.386401212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:06:08.926523826 +0000 UTC m=+551.783607770" watchObservedRunningTime="2026-07-17 11:06:08.930112953 +0000 UTC m=+551.787196897"
Jul 17 11:06:19.894958 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:19.894928    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nclqh"
Jul 17 11:06:57.607785 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:57.607759    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:06:57.610974 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:57.610952    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:06:57.620931 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:57.620906    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:06:57.624023 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:06:57.624005    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:07:21.741232 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:21.741193    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:07:29.273373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:29.273347    2566 generic.go:356] "Generic (PLEG): container finished" podID="93486233-9ab9-4859-ba3e-711e53d4c05a" containerID="d6abe77a5adf1d426da5bf41d4e35f4adca5d46330c5b4cca741361ef4c3b91c" exitCode=0
Jul 17 11:07:29.273679 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:29.273417    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8" event={"ID":"93486233-9ab9-4859-ba3e-711e53d4c05a","Type":"ContainerDied","Data":"d6abe77a5adf1d426da5bf41d4e35f4adca5d46330c5b4cca741361ef4c3b91c"}
Jul 17 11:07:29.273818 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:29.273804    2566 scope.go:117] "RemoveContainer" containerID="d6abe77a5adf1d426da5bf41d4e35f4adca5d46330c5b4cca741361ef4c3b91c"
Jul 17 11:07:29.274768 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:29.274740    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:07:29.274871 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:29.274823    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:07:29.274939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:29.274869    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:07:30.279839 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:30.279806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8" event={"ID":"93486233-9ab9-4859-ba3e-711e53d4c05a","Type":"ContainerStarted","Data":"3f214db8ac42ae41fef9e9a412f73d8eb0e0e27cf577128cec08b10347707284"}
Jul 17 11:07:30.662795 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:30.662759    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:07:30.662988 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:30.662965    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hkwn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 17 11:07:30.663038 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:30.663013    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8" podUID="93486233-9ab9-4859-ba3e-711e53d4c05a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 17 11:07:31.662058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:31.662026    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hkwn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 17 11:07:31.662442 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:31.662081    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8" podUID="93486233-9ab9-4859-ba3e-711e53d4c05a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 17 11:07:32.662329 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:32.662292    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hkwn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 17 11:07:32.662733 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:32.662347    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8" podUID="93486233-9ab9-4859-ba3e-711e53d4c05a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 17 11:07:33.662625 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:33.662592    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hkwn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 17 11:07:33.663033 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:33.662641    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8" podUID="93486233-9ab9-4859-ba3e-711e53d4c05a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 17 11:07:34.666260 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:34.666231    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:07:34.666660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:34.666485    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:07:34.667136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:34.667118    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hkwn8"
Jul 17 11:07:42.175922 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:42.175888    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:07:51.152025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:07:51.151985    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:08:02.905755 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.905720    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"]
Jul 17 11:08:02.909300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.909266    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:02.912895 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.912877    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 11:08:02.913428 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.913415    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-db62k\""
Jul 17 11:08:02.914239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.914225    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 17 11:08:02.914288 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.914244    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 11:08:02.929444 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.929423    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"]
Jul 17 11:08:02.941552 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.941531    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:02.941648 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.941558    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8690b803-7fc8-4a56-914e-7c0dc674275b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:02.941648 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.941616    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:02.941648 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.941639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:02.941778 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.941681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:02.941778 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:02.941734    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf7tj\" (UniqueName: \"kubernetes.io/projected/8690b803-7fc8-4a56-914e-7c0dc674275b-kube-api-access-bf7tj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.042786 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.042753    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf7tj\" (UniqueName: \"kubernetes.io/projected/8690b803-7fc8-4a56-914e-7c0dc674275b-kube-api-access-bf7tj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.042929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.042789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.042929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.042807    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8690b803-7fc8-4a56-914e-7c0dc674275b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.042929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.042863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.042929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.042880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.042929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.042908    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.043238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.043215    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.043278 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.043233    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.043344 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.043286    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.044992 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.044975    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8690b803-7fc8-4a56-914e-7c0dc674275b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.045164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.045147    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8690b803-7fc8-4a56-914e-7c0dc674275b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.058115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.058092    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf7tj\" (UniqueName: \"kubernetes.io/projected/8690b803-7fc8-4a56-914e-7c0dc674275b-kube-api-access-bf7tj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2c4mf\" (UID: \"8690b803-7fc8-4a56-914e-7c0dc674275b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.220163 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.220064    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:03.353831 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.353803    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"]
Jul 17 11:08:03.355511 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:08:03.355484    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8690b803_7fc8_4a56_914e_7c0dc674275b.slice/crio-0c069edd5bc8ead9db7c2a951f326710f744b93cf95ddafd69bc9d22ccf253a8 WatchSource:0}: Error finding container 0c069edd5bc8ead9db7c2a951f326710f744b93cf95ddafd69bc9d22ccf253a8: Status 404 returned error can't find the container with id 0c069edd5bc8ead9db7c2a951f326710f744b93cf95ddafd69bc9d22ccf253a8
Jul 17 11:08:03.356804 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.356784    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 11:08:03.433667 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:03.433636    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf" event={"ID":"8690b803-7fc8-4a56-914e-7c0dc674275b","Type":"ContainerStarted","Data":"0c069edd5bc8ead9db7c2a951f326710f744b93cf95ddafd69bc9d22ccf253a8"}
Jul 17 11:08:04.245627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:04.245588    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:08:05.445669 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:05.445635    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf" event={"ID":"8690b803-7fc8-4a56-914e-7c0dc674275b","Type":"ContainerStarted","Data":"1bdf1598831321074c6f1b942cca7429f87d419673a5f30ce97127fe3f6a457f"}
Jul 17 11:08:05.467353 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:05.467308    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf" podStartSLOduration=2.436255955 podStartE2EDuration="3.467293697s" totalImagesPullingTime="1.031037742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:08:05.465949781 +0000 UTC m=+668.323033753" watchObservedRunningTime="2026-07-17 11:08:05.467293697 +0000 UTC m=+668.324377644"
Jul 17 11:08:12.853346 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:12.853311    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:08:13.220935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:13.220851    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:13.220935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:13.220917    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:13.237367 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:13.237336    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:13.491925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:13.491846    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2c4mf"
Jul 17 11:08:18.954491 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:18.954456    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:08:31.050903 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:08:31.050869    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:09:47.447271 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:09:47.447239    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:09:57.250561 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:09:57.250525    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:10:06.943075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:10:06.943040    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:11:50.654767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:11:50.654677    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:11:57.656394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:11:57.656364    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:11:57.661485 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:11:57.661461    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:11:57.665857 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:11:57.665832    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:11:57.668105 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:11:57.668086    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:12:02.550671 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:12:02.550636    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:12:37.646296 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:12:37.646256    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:12:44.975550 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:12:44.975508    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:12:54.149502 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:12:54.149472    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:13:01.560374 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:13:01.560340    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:13:11.251859 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:13:11.251821    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:13:20.482229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:13:20.482152    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:13:32.145245 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:13:32.145201    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:13:41.559490 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:13:41.559443    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:14:07.968596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:14:07.968563    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:14:17.054196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:14:17.054158    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:14:42.364612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:14:42.364575    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:14:52.452288 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:14:52.452212    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:15:00.145343 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:00.145309    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r"]
Jul 17 11:15:00.148521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:00.148504    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r"
Jul 17 11:15:00.151242 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:00.151220    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:15:00.157920 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:00.157903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r"
Jul 17 11:15:00.166192 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:00.166173    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r"]
Jul 17 11:15:00.295605 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:00.295581    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r"]
Jul 17 11:15:00.297880 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:15:00.297854    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d45de2_506a_4b15_b0be_0ee8c281c36f.slice/crio-69a1e9e19e125426ffd414b22eadda31adb50665c85d92693a3567b0b92b92f3 WatchSource:0}: Error finding container 69a1e9e19e125426ffd414b22eadda31adb50665c85d92693a3567b0b92b92f3: Status 404 returned error can't find the container with id 69a1e9e19e125426ffd414b22eadda31adb50665c85d92693a3567b0b92b92f3
Jul 17 11:15:00.300176 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:00.300160    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 11:15:00.342746 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:00.342695    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r" event={"ID":"93d45de2-506a-4b15-b0be-0ee8c281c36f","Type":"ContainerStarted","Data":"69a1e9e19e125426ffd414b22eadda31adb50665c85d92693a3567b0b92b92f3"}
Jul 17 11:15:03.358776 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:03.358749    2566 generic.go:356] "Generic (PLEG): container finished" podID="93d45de2-506a-4b15-b0be-0ee8c281c36f" containerID="d3225140d15e9c6c389dc9bb4d4da8dac66133bb8b06db99f916cb0a3c12c319" exitCode=0
Jul 17 11:15:03.359086 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:03.358827    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r" event={"ID":"93d45de2-506a-4b15-b0be-0ee8c281c36f","Type":"ContainerDied","Data":"d3225140d15e9c6c389dc9bb4d4da8dac66133bb8b06db99f916cb0a3c12c319"}
Jul 17 11:15:04.500008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:04.499969    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r"
Jul 17 11:15:05.369423 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:05.369396    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r"
Jul 17 11:15:05.369596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:05.369434    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738115-xvs6r" event={"ID":"93d45de2-506a-4b15-b0be-0ee8c281c36f","Type":"ContainerDied","Data":"69a1e9e19e125426ffd414b22eadda31adb50665c85d92693a3567b0b92b92f3"}
Jul 17 11:15:05.369596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:05.369479    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69a1e9e19e125426ffd414b22eadda31adb50665c85d92693a3567b0b92b92f3"
Jul 17 11:15:08.174436 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:08.174404    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:15:19.059135 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:19.059103    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:15:28.561456 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:28.561422    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:15:37.850830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:37.850799    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:15:47.181117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:15:47.176844    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:16:57.692619 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:16:57.692592    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:16:57.693149 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:16:57.692757    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:16:57.695671 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:16:57.695651    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:16:57.695884 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:16:57.695868    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:17:36.748390 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:17:36.748348    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:17:45.643695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:17:45.643660    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:17:58.146862 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:17:58.146782    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:18:09.656952 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:18:09.656916    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:18:20.179561 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:18:20.179524    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:18:32.354748 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:18:32.354695    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:18:40.245428 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:18:40.245387    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:18:51.038753 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:18:51.038720    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:19:03.348242 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:19:03.348204    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:19:12.150673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:19:12.150636    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:19:22.657357 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:19:22.657321    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:19:46.950455 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:19:46.950421    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:19:54.687853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:19:54.687816    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:20:04.444312 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:20:04.444275    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:20:14.846609 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:20:14.846573    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:20:24.451924 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:20:24.451893    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:20:37.547834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:20:37.547795    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:20:46.751638 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:20:46.751608    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:20:56.265502 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:20:56.265421    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:21:03.752157 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:03.752123    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:21:14.852465 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:14.852430    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:21:23.902551 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:23.902514    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:21:36.860682 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:36.860648    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:21:45.944258 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:45.944223    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:21:56.643929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:56.643895    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:21:57.722114 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:57.722089    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:21:57.725044 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:57.725022    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:21:57.725524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:57.725502    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:21:57.728473 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:21:57.728449    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:22:07.051302 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:22:07.051263    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:22:19.754489 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:22:19.754452    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:22:29.736911 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:22:29.736877    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:22:47.549448 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:22:47.549418    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:22:59.549598 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:22:59.549562    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:23:10.444621 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:10.444583    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:23:21.142444 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.142413    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"]
Jul 17 11:23:21.142912 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.142809    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93d45de2-506a-4b15-b0be-0ee8c281c36f" containerName="cleanup"
Jul 17 11:23:21.142912 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.142821    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="93d45de2-506a-4b15-b0be-0ee8c281c36f" containerName="cleanup"
Jul 17 11:23:21.142912 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.142875    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="93d45de2-506a-4b15-b0be-0ee8c281c36f" containerName="cleanup"
Jul 17 11:23:21.146175 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.146154    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.148802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.148700    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-829f05b9-openshift-default-dockercfg-cpvz8\""
Jul 17 11:23:21.166514 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.166491    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"]
Jul 17 11:23:21.263409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.263379    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-data\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.263560 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.263426    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-podinfo\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.263560 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.263461    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-envoy\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.263560 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.263480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-token\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.263560 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.263496    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istiod-ca-cert\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.263689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.263548    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-credential-socket\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.263689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.263583    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5shg\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-kube-api-access-n5shg\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.263689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.263605    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-certs\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.263689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.263625    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-socket\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364372 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-data\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364381    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-podinfo\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364418    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-envoy\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364435    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-token\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364451    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istiod-ca-cert\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364742 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-credential-socket\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364742 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364733    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-data\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364860 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5shg\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-kube-api-access-n5shg\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364860 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364796    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-certs\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.364860 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.364827    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-socket\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.365135 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.365110    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-credential-socket\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.365135 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.365128    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-certs\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.365309 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.365276    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-socket\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.365309 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.365296    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istiod-ca-cert\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.366784 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.366764    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-envoy\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.366985 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.366968    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-podinfo\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.374267 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.374243    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-token\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.374364 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.374285    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5shg\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-kube-api-access-n5shg\") pod \"e2e-ait-829f05b9-openshift-default-658985dd69-hh85t\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") " pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.458828 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.458749    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:21.592438 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.592407    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"]
Jul 17 11:23:21.593932 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:23:21.593906    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e517352_b770_41c8_b0d7_f10dbd5aa71d.slice/crio-9de14b02a78cb8bddc77b09a91ffd00bfd0bf9a38c8e1856a2a7de32755cbf19 WatchSource:0}: Error finding container 9de14b02a78cb8bddc77b09a91ffd00bfd0bf9a38c8e1856a2a7de32755cbf19: Status 404 returned error can't find the container with id 9de14b02a78cb8bddc77b09a91ffd00bfd0bf9a38c8e1856a2a7de32755cbf19
Jul 17 11:23:21.595925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.595886    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:23:21.596026 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.595952    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:23:21.596026 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.595980    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:23:21.682699 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.682656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" event={"ID":"8e517352-b770-41c8-b0d7-f10dbd5aa71d","Type":"ContainerStarted","Data":"b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361"}
Jul 17 11:23:21.682885 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.682748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" event={"ID":"8e517352-b770-41c8-b0d7-f10dbd5aa71d","Type":"ContainerStarted","Data":"9de14b02a78cb8bddc77b09a91ffd00bfd0bf9a38c8e1856a2a7de32755cbf19"}
Jul 17 11:23:21.704115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:21.704056    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" podStartSLOduration=0.70403513 podStartE2EDuration="704.03513ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:23:21.703094815 +0000 UTC m=+1584.560178772" watchObservedRunningTime="2026-07-17 11:23:21.70403513 +0000 UTC m=+1584.561119065"
Jul 17 11:23:22.459629 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:22.459596    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:22.464394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:22.464372    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:22.688816 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:22.688777    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:22.689933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:22.689912    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:27.426633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.426599    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"]
Jul 17 11:23:27.430137 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.430121    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"
Jul 17 11:23:27.432519 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.432495    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:23:27.432637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.432617    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:23:27.441941 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.441918    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"]
Jul 17 11:23:27.519457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.519428    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a17024-6623-4bd6-ad87-df37f532e8d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6\" (UID: \"84a17024-6623-4bd6-ad87-df37f532e8d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"
Jul 17 11:23:27.619892 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.619866    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a17024-6623-4bd6-ad87-df37f532e8d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6\" (UID: \"84a17024-6623-4bd6-ad87-df37f532e8d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"
Jul 17 11:23:27.620695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.620672    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a17024-6623-4bd6-ad87-df37f532e8d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6\" (UID: \"84a17024-6623-4bd6-ad87-df37f532e8d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"
Jul 17 11:23:27.741053 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.740985    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"
Jul 17 11:23:27.870654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:27.870631    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"]
Jul 17 11:23:27.872562 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:23:27.872534    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a17024_6623_4bd6_ad87_df37f532e8d5.slice/crio-8adc2957d2912db110b663c0a9194fcb1dfda0f14796ab94f7deee4ca3b9ec81 WatchSource:0}: Error finding container 8adc2957d2912db110b663c0a9194fcb1dfda0f14796ab94f7deee4ca3b9ec81: Status 404 returned error can't find the container with id 8adc2957d2912db110b663c0a9194fcb1dfda0f14796ab94f7deee4ca3b9ec81
Jul 17 11:23:28.717005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:28.716922    2566 generic.go:356] "Generic (PLEG): container finished" podID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerID="4d0a5a3767a9cbe70941abfc222d5596c217b30eab7e0e05b89cc630657daad3" exitCode=7
Jul 17 11:23:28.717005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:28.716991    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6" event={"ID":"84a17024-6623-4bd6-ad87-df37f532e8d5","Type":"ContainerDied","Data":"4d0a5a3767a9cbe70941abfc222d5596c217b30eab7e0e05b89cc630657daad3"}
Jul 17 11:23:28.717407 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:28.717037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6" event={"ID":"84a17024-6623-4bd6-ad87-df37f532e8d5","Type":"ContainerStarted","Data":"8adc2957d2912db110b663c0a9194fcb1dfda0f14796ab94f7deee4ca3b9ec81"}
Jul 17 11:23:28.717407 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:28.717132    2566 scope.go:117] "RemoveContainer" containerID="4d0a5a3767a9cbe70941abfc222d5596c217b30eab7e0e05b89cc630657daad3"
Jul 17 11:23:29.722821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:29.722737    2566 generic.go:356] "Generic (PLEG): container finished" podID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerID="5353c658a6a0d23725359039acfc77563a055a151f709045ad1d6ded76bfa32d" exitCode=7
Jul 17 11:23:29.723191 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:29.722828    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6" event={"ID":"84a17024-6623-4bd6-ad87-df37f532e8d5","Type":"ContainerDied","Data":"5353c658a6a0d23725359039acfc77563a055a151f709045ad1d6ded76bfa32d"}
Jul 17 11:23:29.723191 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:29.722886    2566 scope.go:117] "RemoveContainer" containerID="4d0a5a3767a9cbe70941abfc222d5596c217b30eab7e0e05b89cc630657daad3"
Jul 17 11:23:29.723191 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:29.722986    2566 scope.go:117] "RemoveContainer" containerID="5353c658a6a0d23725359039acfc77563a055a151f709045ad1d6ded76bfa32d"
Jul 17 11:23:29.723314 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:23:29.723185    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6_odh-ai-gateway-infra(84a17024-6623-4bd6-ad87-df37f532e8d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5"
Jul 17 11:23:29.855554 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:29.855521    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:23:30.729962 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:30.729934    2566 scope.go:117] "RemoveContainer" containerID="5353c658a6a0d23725359039acfc77563a055a151f709045ad1d6ded76bfa32d"
Jul 17 11:23:30.730442 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:23:30.730141    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6_odh-ai-gateway-infra(84a17024-6623-4bd6-ad87-df37f532e8d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5"
Jul 17 11:23:37.935031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:37.934233    2566 patch_prober.go:28] interesting pod/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": read tcp 10.132.0.2:52950->10.132.0.37:15021: read: connection reset by peer" start-of-body=
Jul 17 11:23:37.935031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:37.934303    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": read tcp 10.132.0.2:52950->10.132.0.37:15021: read: connection reset by peer"
Jul 17 11:23:38.778235 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:38.778186    2566 generic.go:356] "Generic (PLEG): container finished" podID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerID="b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361" exitCode=0
Jul 17 11:23:38.778398 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:38.778263    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" event={"ID":"8e517352-b770-41c8-b0d7-f10dbd5aa71d","Type":"ContainerDied","Data":"b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361"}
Jul 17 11:23:38.778629 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:38.778616    2566 scope.go:117] "RemoveContainer" containerID="b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361"
Jul 17 11:23:38.779685 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:38.779651    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:23:38.779799 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:38.779765    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:23:38.779850 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:38.779825    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:23:39.784392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:39.784354    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" event={"ID":"8e517352-b770-41c8-b0d7-f10dbd5aa71d","Type":"ContainerStarted","Data":"e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44"}
Jul 17 11:23:40.459560 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:40.459524    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:40.459758 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:40.459730    2566 patch_prober.go:28] interesting pod/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 17 11:23:40.459989 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:40.459771    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 17 11:23:41.459566 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:41.459532    2566 patch_prober.go:28] interesting pod/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 17 11:23:41.459939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:41.459589    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 17 11:23:42.459161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:42.459130    2566 patch_prober.go:28] interesting pod/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 17 11:23:42.459338 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:42.459179    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 17 11:23:43.459597 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:43.459543    2566 patch_prober.go:28] interesting pod/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 17 11:23:43.460028 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:43.459622    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 17 11:23:44.462433 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:44.462401    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:44.462821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:44.462683    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:44.463442 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:44.463423    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:23:44.652623 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:44.652593    2566 scope.go:117] "RemoveContainer" containerID="5353c658a6a0d23725359039acfc77563a055a151f709045ad1d6ded76bfa32d"
Jul 17 11:23:45.819350 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:45.819316    2566 generic.go:356] "Generic (PLEG): container finished" podID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerID="fbd00b3804f34a639a26b9da58ff21f3da2204b83de2e1c6187712b4624fa875" exitCode=7
Jul 17 11:23:45.819752 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:45.819377    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6" event={"ID":"84a17024-6623-4bd6-ad87-df37f532e8d5","Type":"ContainerDied","Data":"fbd00b3804f34a639a26b9da58ff21f3da2204b83de2e1c6187712b4624fa875"}
Jul 17 11:23:45.819752 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:45.819433    2566 scope.go:117] "RemoveContainer" containerID="5353c658a6a0d23725359039acfc77563a055a151f709045ad1d6ded76bfa32d"
Jul 17 11:23:45.819878 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:45.819837    2566 scope.go:117] "RemoveContainer" containerID="fbd00b3804f34a639a26b9da58ff21f3da2204b83de2e1c6187712b4624fa875"
Jul 17 11:23:45.820057 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:23:45.820032    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6_odh-ai-gateway-infra(84a17024-6623-4bd6-ad87-df37f532e8d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5"
Jul 17 11:23:46.849570 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:46.849536    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"]
Jul 17 11:23:46.980620 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:46.980599    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"
Jul 17 11:23:47.008297 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:47.008271    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a17024-6623-4bd6-ad87-df37f532e8d5-maas-api-service-ca\") pod \"84a17024-6623-4bd6-ad87-df37f532e8d5\" (UID: \"84a17024-6623-4bd6-ad87-df37f532e8d5\") "
Jul 17 11:23:47.008620 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:47.008597    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a17024-6623-4bd6-ad87-df37f532e8d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "84a17024-6623-4bd6-ad87-df37f532e8d5" (UID: "84a17024-6623-4bd6-ad87-df37f532e8d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:23:47.109346 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:47.109288    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a17024-6623-4bd6-ad87-df37f532e8d5-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:23:47.831520 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:47.831484    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6" event={"ID":"84a17024-6623-4bd6-ad87-df37f532e8d5","Type":"ContainerDied","Data":"8adc2957d2912db110b663c0a9194fcb1dfda0f14796ab94f7deee4ca3b9ec81"}
Jul 17 11:23:47.831520 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:47.831512    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"
Jul 17 11:23:47.831752 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:47.831565    2566 scope.go:117] "RemoveContainer" containerID="fbd00b3804f34a639a26b9da58ff21f3da2204b83de2e1c6187712b4624fa875"
Jul 17 11:23:47.854340 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:47.854317    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"]
Jul 17 11:23:47.860229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:47.860209    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xn6f6"]
Jul 17 11:23:49.657628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:23:49.657595    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" path="/var/lib/kubelet/pods/84a17024-6623-4bd6-ad87-df37f532e8d5/volumes"
Jul 17 11:24:27.483695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.483665    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"]
Jul 17 11:24:27.484140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.484031    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerName="revoke-keys"
Jul 17 11:24:27.484140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.484042    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerName="revoke-keys"
Jul 17 11:24:27.484140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.484064    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerName="revoke-keys"
Jul 17 11:24:27.484140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.484069    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerName="revoke-keys"
Jul 17 11:24:27.484140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.484079    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerName="revoke-keys"
Jul 17 11:24:27.484140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.484085    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerName="revoke-keys"
Jul 17 11:24:27.484369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.484176    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerName="revoke-keys"
Jul 17 11:24:27.484369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.484184    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerName="revoke-keys"
Jul 17 11:24:27.487212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.487195    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"
Jul 17 11:24:27.489915 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.489892    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:24:27.489915 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.489894    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:24:27.495943 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.495923    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"]
Jul 17 11:24:27.658580 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.658550    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f85ca33-3fc4-4969-b315-a58ea6fb78d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-v5smm\" (UID: \"4f85ca33-3fc4-4969-b315-a58ea6fb78d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"
Jul 17 11:24:27.760064 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.759989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f85ca33-3fc4-4969-b315-a58ea6fb78d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-v5smm\" (UID: \"4f85ca33-3fc4-4969-b315-a58ea6fb78d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"
Jul 17 11:24:27.760540 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.760521    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f85ca33-3fc4-4969-b315-a58ea6fb78d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-v5smm\" (UID: \"4f85ca33-3fc4-4969-b315-a58ea6fb78d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"
Jul 17 11:24:27.798199 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.798178    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"
Jul 17 11:24:27.927205 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:27.927183    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"]
Jul 17 11:24:27.929372 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:24:27.929342    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f85ca33_3fc4_4969_b315_a58ea6fb78d8.slice/crio-d4374521df0e5c077fde6832f8f1e91437297f5605e478e937949681161d323d WatchSource:0}: Error finding container d4374521df0e5c077fde6832f8f1e91437297f5605e478e937949681161d323d: Status 404 returned error can't find the container with id d4374521df0e5c077fde6832f8f1e91437297f5605e478e937949681161d323d
Jul 17 11:24:28.019484 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:28.019417    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" event={"ID":"4f85ca33-3fc4-4969-b315-a58ea6fb78d8","Type":"ContainerStarted","Data":"d4374521df0e5c077fde6832f8f1e91437297f5605e478e937949681161d323d"}
Jul 17 11:24:29.025371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:29.025281    2566 generic.go:356] "Generic (PLEG): container finished" podID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerID="262d9e56e0eb668a747fc70d7c61a8bcb80cfe88f8732fd0dee081e0a1a7bbdf" exitCode=7
Jul 17 11:24:29.025371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:29.025356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" event={"ID":"4f85ca33-3fc4-4969-b315-a58ea6fb78d8","Type":"ContainerDied","Data":"262d9e56e0eb668a747fc70d7c61a8bcb80cfe88f8732fd0dee081e0a1a7bbdf"}
Jul 17 11:24:29.025790 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:29.025526    2566 scope.go:117] "RemoveContainer" containerID="262d9e56e0eb668a747fc70d7c61a8bcb80cfe88f8732fd0dee081e0a1a7bbdf"
Jul 17 11:24:30.032109 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:30.032015    2566 generic.go:356] "Generic (PLEG): container finished" podID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerID="aee14885a9a414b4b28230bcf69e65bd5e7c4f51ea489b4e201f79898d8b41d3" exitCode=7
Jul 17 11:24:30.032557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:30.032106    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" event={"ID":"4f85ca33-3fc4-4969-b315-a58ea6fb78d8","Type":"ContainerDied","Data":"aee14885a9a414b4b28230bcf69e65bd5e7c4f51ea489b4e201f79898d8b41d3"}
Jul 17 11:24:30.032557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:30.032167    2566 scope.go:117] "RemoveContainer" containerID="262d9e56e0eb668a747fc70d7c61a8bcb80cfe88f8732fd0dee081e0a1a7bbdf"
Jul 17 11:24:30.032557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:30.032389    2566 scope.go:117] "RemoveContainer" containerID="aee14885a9a414b4b28230bcf69e65bd5e7c4f51ea489b4e201f79898d8b41d3"
Jul 17 11:24:30.032735 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:24:30.032602    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-v5smm_odh-ai-gateway-infra(4f85ca33-3fc4-4969-b315-a58ea6fb78d8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8"
Jul 17 11:24:31.038040 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:31.038014    2566 scope.go:117] "RemoveContainer" containerID="aee14885a9a414b4b28230bcf69e65bd5e7c4f51ea489b4e201f79898d8b41d3"
Jul 17 11:24:31.038403 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:24:31.038194    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-v5smm_odh-ai-gateway-infra(4f85ca33-3fc4-4969-b315-a58ea6fb78d8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8"
Jul 17 11:24:44.651596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:44.651568    2566 scope.go:117] "RemoveContainer" containerID="aee14885a9a414b4b28230bcf69e65bd5e7c4f51ea489b4e201f79898d8b41d3"
Jul 17 11:24:45.103890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:45.103858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" event={"ID":"4f85ca33-3fc4-4969-b315-a58ea6fb78d8","Type":"ContainerStarted","Data":"bfe6c324c1dadda48edb52bfca414b06fd51c91a8bc8515a7013c9877e208164"}
Jul 17 11:24:45.127392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:45.127348    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" podStartSLOduration=18.12733256 podStartE2EDuration="18.12733256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:24:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:24:45.126187263 +0000 UTC m=+1667.983271208" watchObservedRunningTime="2026-07-17 11:24:45.12733256 +0000 UTC m=+1667.984416504"
Jul 17 11:24:46.109884 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:46.109850    2566 generic.go:356] "Generic (PLEG): container finished" podID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerID="bfe6c324c1dadda48edb52bfca414b06fd51c91a8bc8515a7013c9877e208164" exitCode=7
Jul 17 11:24:46.110256 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:46.109926    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" event={"ID":"4f85ca33-3fc4-4969-b315-a58ea6fb78d8","Type":"ContainerDied","Data":"bfe6c324c1dadda48edb52bfca414b06fd51c91a8bc8515a7013c9877e208164"}
Jul 17 11:24:46.110256 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:46.109984    2566 scope.go:117] "RemoveContainer" containerID="aee14885a9a414b4b28230bcf69e65bd5e7c4f51ea489b4e201f79898d8b41d3"
Jul 17 11:24:46.110256 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:46.110223    2566 scope.go:117] "RemoveContainer" containerID="bfe6c324c1dadda48edb52bfca414b06fd51c91a8bc8515a7013c9877e208164"
Jul 17 11:24:46.110463 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:24:46.110422    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-v5smm_odh-ai-gateway-infra(4f85ca33-3fc4-4969-b315-a58ea6fb78d8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8"
Jul 17 11:24:46.136782 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:46.136757    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"]
Jul 17 11:24:47.247302 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:47.247278    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"
Jul 17 11:24:47.319828 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:47.319798    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f85ca33-3fc4-4969-b315-a58ea6fb78d8-maas-api-service-ca\") pod \"4f85ca33-3fc4-4969-b315-a58ea6fb78d8\" (UID: \"4f85ca33-3fc4-4969-b315-a58ea6fb78d8\") "
Jul 17 11:24:47.320160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:47.320136    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f85ca33-3fc4-4969-b315-a58ea6fb78d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f85ca33-3fc4-4969-b315-a58ea6fb78d8" (UID: "4f85ca33-3fc4-4969-b315-a58ea6fb78d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:24:47.421315 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:47.421241    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f85ca33-3fc4-4969-b315-a58ea6fb78d8-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:24:48.120536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:48.120497    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"
Jul 17 11:24:48.120536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:48.120529    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm" event={"ID":"4f85ca33-3fc4-4969-b315-a58ea6fb78d8","Type":"ContainerDied","Data":"d4374521df0e5c077fde6832f8f1e91437297f5605e478e937949681161d323d"}
Jul 17 11:24:48.120791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:48.120592    2566 scope.go:117] "RemoveContainer" containerID="bfe6c324c1dadda48edb52bfca414b06fd51c91a8bc8515a7013c9877e208164"
Jul 17 11:24:48.140033 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:48.140003    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"]
Jul 17 11:24:48.145144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:48.145121    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-v5smm"]
Jul 17 11:24:49.657748 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:24:49.657696    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" path="/var/lib/kubelet/pods/4f85ca33-3fc4-4969-b315-a58ea6fb78d8/volumes"
Jul 17 11:25:27.613623 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.613583    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"]
Jul 17 11:25:27.614204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.614143    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerName="revoke-keys"
Jul 17 11:25:27.614204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.614164    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerName="revoke-keys"
Jul 17 11:25:27.614204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.614182    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerName="revoke-keys"
Jul 17 11:25:27.614204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.614191    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerName="revoke-keys"
Jul 17 11:25:27.614421 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.614280    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerName="revoke-keys"
Jul 17 11:25:27.614421 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.614299    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerName="revoke-keys"
Jul 17 11:25:27.614421 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.614310    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a17024-6623-4bd6-ad87-df37f532e8d5" containerName="revoke-keys"
Jul 17 11:25:27.617322 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.617300    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"
Jul 17 11:25:27.619898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.619872    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:25:27.620003 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.619907    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:25:27.626511 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.626492    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"]
Jul 17 11:25:27.678582 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.678552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d73d59d7-4a9d-4001-bcf7-600d0120341e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr\" (UID: \"d73d59d7-4a9d-4001-bcf7-600d0120341e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"
Jul 17 11:25:27.779887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.779863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d73d59d7-4a9d-4001-bcf7-600d0120341e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr\" (UID: \"d73d59d7-4a9d-4001-bcf7-600d0120341e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"
Jul 17 11:25:27.780447 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.780428    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d73d59d7-4a9d-4001-bcf7-600d0120341e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr\" (UID: \"d73d59d7-4a9d-4001-bcf7-600d0120341e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"
Jul 17 11:25:27.929463 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:27.929391    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"
Jul 17 11:25:28.057396 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:28.057372    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"]
Jul 17 11:25:28.059267 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:25:28.059239    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73d59d7_4a9d_4001_bcf7_600d0120341e.slice/crio-d7bebabc30fc57b36306b393b7a28d8dae929996da5c2cabc33e41479b8ce7ba WatchSource:0}: Error finding container d7bebabc30fc57b36306b393b7a28d8dae929996da5c2cabc33e41479b8ce7ba: Status 404 returned error can't find the container with id d7bebabc30fc57b36306b393b7a28d8dae929996da5c2cabc33e41479b8ce7ba
Jul 17 11:25:28.310215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:28.310185    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" event={"ID":"d73d59d7-4a9d-4001-bcf7-600d0120341e","Type":"ContainerStarted","Data":"e2e4db7ff32112d264d1492170a9b15109df3595598b062292ee275acc97cbcc"}
Jul 17 11:25:28.310336 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:28.310260    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" event={"ID":"d73d59d7-4a9d-4001-bcf7-600d0120341e","Type":"ContainerStarted","Data":"d7bebabc30fc57b36306b393b7a28d8dae929996da5c2cabc33e41479b8ce7ba"}
Jul 17 11:25:28.327594 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:28.327550    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" podStartSLOduration=1.327536555 podStartE2EDuration="1.327536555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:25:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:25:28.325928819 +0000 UTC m=+1711.183012764" watchObservedRunningTime="2026-07-17 11:25:28.327536555 +0000 UTC m=+1711.184620532"
Jul 17 11:25:29.316623 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:29.316585    2566 generic.go:356] "Generic (PLEG): container finished" podID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerID="e2e4db7ff32112d264d1492170a9b15109df3595598b062292ee275acc97cbcc" exitCode=7
Jul 17 11:25:29.317110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:29.316630    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" event={"ID":"d73d59d7-4a9d-4001-bcf7-600d0120341e","Type":"ContainerDied","Data":"e2e4db7ff32112d264d1492170a9b15109df3595598b062292ee275acc97cbcc"}
Jul 17 11:25:29.317110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:29.316889    2566 scope.go:117] "RemoveContainer" containerID="e2e4db7ff32112d264d1492170a9b15109df3595598b062292ee275acc97cbcc"
Jul 17 11:25:30.323505 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:30.323471    2566 generic.go:356] "Generic (PLEG): container finished" podID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerID="3d5ae2515afc47a2a9529c8d791f1aa39b52ae2fa1438355ad85c951b15ff9d1" exitCode=7
Jul 17 11:25:30.323975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:30.323520    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" event={"ID":"d73d59d7-4a9d-4001-bcf7-600d0120341e","Type":"ContainerDied","Data":"3d5ae2515afc47a2a9529c8d791f1aa39b52ae2fa1438355ad85c951b15ff9d1"}
Jul 17 11:25:30.323975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:30.323577    2566 scope.go:117] "RemoveContainer" containerID="e2e4db7ff32112d264d1492170a9b15109df3595598b062292ee275acc97cbcc"
Jul 17 11:25:30.323975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:30.323785    2566 scope.go:117] "RemoveContainer" containerID="3d5ae2515afc47a2a9529c8d791f1aa39b52ae2fa1438355ad85c951b15ff9d1"
Jul 17 11:25:30.324090 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:25:30.323990    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr_odh-ai-gateway-infra(d73d59d7-4a9d-4001-bcf7-600d0120341e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e"
Jul 17 11:25:31.330260 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:31.330227    2566 scope.go:117] "RemoveContainer" containerID="3d5ae2515afc47a2a9529c8d791f1aa39b52ae2fa1438355ad85c951b15ff9d1"
Jul 17 11:25:31.330744 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:25:31.330485    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr_odh-ai-gateway-infra(d73d59d7-4a9d-4001-bcf7-600d0120341e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e"
Jul 17 11:25:44.652007 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:44.651979    2566 scope.go:117] "RemoveContainer" containerID="3d5ae2515afc47a2a9529c8d791f1aa39b52ae2fa1438355ad85c951b15ff9d1"
Jul 17 11:25:45.409158 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:45.409128    2566 generic.go:356] "Generic (PLEG): container finished" podID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerID="69566e63e291828456f57566817f1fa21449dd238525bdbba599523fd02141f3" exitCode=7
Jul 17 11:25:45.409324 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:45.409184    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" event={"ID":"d73d59d7-4a9d-4001-bcf7-600d0120341e","Type":"ContainerDied","Data":"69566e63e291828456f57566817f1fa21449dd238525bdbba599523fd02141f3"}
Jul 17 11:25:45.409324 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:45.409230    2566 scope.go:117] "RemoveContainer" containerID="3d5ae2515afc47a2a9529c8d791f1aa39b52ae2fa1438355ad85c951b15ff9d1"
Jul 17 11:25:45.409510 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:45.409490    2566 scope.go:117] "RemoveContainer" containerID="69566e63e291828456f57566817f1fa21449dd238525bdbba599523fd02141f3"
Jul 17 11:25:45.409728 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:25:45.409687    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr_odh-ai-gateway-infra(d73d59d7-4a9d-4001-bcf7-600d0120341e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e"
Jul 17 11:25:46.455264 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:46.455222    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"]
Jul 17 11:25:46.590558 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:46.590528    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"
Jul 17 11:25:46.746547 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:46.746457    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d73d59d7-4a9d-4001-bcf7-600d0120341e-maas-api-service-ca\") pod \"d73d59d7-4a9d-4001-bcf7-600d0120341e\" (UID: \"d73d59d7-4a9d-4001-bcf7-600d0120341e\") "
Jul 17 11:25:46.746858 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:46.746837    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d73d59d7-4a9d-4001-bcf7-600d0120341e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d73d59d7-4a9d-4001-bcf7-600d0120341e" (UID: "d73d59d7-4a9d-4001-bcf7-600d0120341e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:25:46.847146 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:46.847110    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d73d59d7-4a9d-4001-bcf7-600d0120341e-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:25:47.421507 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:47.421468    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr" event={"ID":"d73d59d7-4a9d-4001-bcf7-600d0120341e","Type":"ContainerDied","Data":"d7bebabc30fc57b36306b393b7a28d8dae929996da5c2cabc33e41479b8ce7ba"}
Jul 17 11:25:47.421507 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:47.421509    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"
Jul 17 11:25:47.421764 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:47.421533    2566 scope.go:117] "RemoveContainer" containerID="69566e63e291828456f57566817f1fa21449dd238525bdbba599523fd02141f3"
Jul 17 11:25:47.445915 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:47.445888    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"]
Jul 17 11:25:47.448971 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:47.448952    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-tj4mr"]
Jul 17 11:25:47.663558 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:25:47.663524    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" path="/var/lib/kubelet/pods/d73d59d7-4a9d-4001-bcf7-600d0120341e/volumes"
Jul 17 11:26:17.670097 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670064    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"]
Jul 17 11:26:17.670555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670432    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerName="revoke-keys"
Jul 17 11:26:17.670555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670446    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerName="revoke-keys"
Jul 17 11:26:17.670555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670468    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerName="revoke-keys"
Jul 17 11:26:17.670555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670474    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerName="revoke-keys"
Jul 17 11:26:17.670555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670479    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerName="revoke-keys"
Jul 17 11:26:17.670555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670485    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerName="revoke-keys"
Jul 17 11:26:17.670555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670552    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerName="revoke-keys"
Jul 17 11:26:17.670802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670563    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f85ca33-3fc4-4969-b315-a58ea6fb78d8" containerName="revoke-keys"
Jul 17 11:26:17.670802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.670570    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerName="revoke-keys"
Jul 17 11:26:17.673278 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.673261    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"
Jul 17 11:26:17.676147 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.676127    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:26:17.676508 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.676493    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:26:17.683004 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.682982    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"]
Jul 17 11:26:17.707135 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.707107    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86fb5438-8846-4d53-9010-6428f426f495-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-6x65d\" (UID: \"86fb5438-8846-4d53-9010-6428f426f495\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"
Jul 17 11:26:17.807940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.807904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86fb5438-8846-4d53-9010-6428f426f495-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-6x65d\" (UID: \"86fb5438-8846-4d53-9010-6428f426f495\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"
Jul 17 11:26:17.808517 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.808494    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86fb5438-8846-4d53-9010-6428f426f495-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-6x65d\" (UID: \"86fb5438-8846-4d53-9010-6428f426f495\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"
Jul 17 11:26:17.983559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:17.983475    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"
Jul 17 11:26:18.113399 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:18.113367    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"]
Jul 17 11:26:18.116004 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:26:18.115979    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86fb5438_8846_4d53_9010_6428f426f495.slice/crio-12747eb179559f1e765bc9c49a4b3e30d23decab4a489f0b87c210b0f9acf242 WatchSource:0}: Error finding container 12747eb179559f1e765bc9c49a4b3e30d23decab4a489f0b87c210b0f9acf242: Status 404 returned error can't find the container with id 12747eb179559f1e765bc9c49a4b3e30d23decab4a489f0b87c210b0f9acf242
Jul 17 11:26:18.572935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:18.572896    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d" event={"ID":"86fb5438-8846-4d53-9010-6428f426f495","Type":"ContainerStarted","Data":"bc4ca37ad76a2ef87bea21bc381f36c59ce0cfb6f2563adacc41234f937d37c1"}
Jul 17 11:26:18.573149 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:18.572990    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d" event={"ID":"86fb5438-8846-4d53-9010-6428f426f495","Type":"ContainerStarted","Data":"12747eb179559f1e765bc9c49a4b3e30d23decab4a489f0b87c210b0f9acf242"}
Jul 17 11:26:19.579508 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:19.579470    2566 generic.go:356] "Generic (PLEG): container finished" podID="86fb5438-8846-4d53-9010-6428f426f495" containerID="bc4ca37ad76a2ef87bea21bc381f36c59ce0cfb6f2563adacc41234f937d37c1" exitCode=7
Jul 17 11:26:19.579954 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:19.579556    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d" event={"ID":"86fb5438-8846-4d53-9010-6428f426f495","Type":"ContainerDied","Data":"bc4ca37ad76a2ef87bea21bc381f36c59ce0cfb6f2563adacc41234f937d37c1"}
Jul 17 11:26:19.579954 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:19.579764    2566 scope.go:117] "RemoveContainer" containerID="bc4ca37ad76a2ef87bea21bc381f36c59ce0cfb6f2563adacc41234f937d37c1"
Jul 17 11:26:20.586769 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:20.586737    2566 generic.go:356] "Generic (PLEG): container finished" podID="86fb5438-8846-4d53-9010-6428f426f495" containerID="ba8d9ff90b5b1d4aec32b3a5d784b754f91c1d635dcd04309cfe080048c0f90a" exitCode=7
Jul 17 11:26:20.587231 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:20.586820    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d" event={"ID":"86fb5438-8846-4d53-9010-6428f426f495","Type":"ContainerDied","Data":"ba8d9ff90b5b1d4aec32b3a5d784b754f91c1d635dcd04309cfe080048c0f90a"}
Jul 17 11:26:20.587231 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:20.586882    2566 scope.go:117] "RemoveContainer" containerID="bc4ca37ad76a2ef87bea21bc381f36c59ce0cfb6f2563adacc41234f937d37c1"
Jul 17 11:26:20.587231 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:20.587105    2566 scope.go:117] "RemoveContainer" containerID="ba8d9ff90b5b1d4aec32b3a5d784b754f91c1d635dcd04309cfe080048c0f90a"
Jul 17 11:26:20.587368 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:26:20.587340    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-6x65d_odh-ai-gateway-infra(86fb5438-8846-4d53-9010-6428f426f495)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d" podUID="86fb5438-8846-4d53-9010-6428f426f495"
Jul 17 11:26:21.593451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:21.593426    2566 scope.go:117] "RemoveContainer" containerID="ba8d9ff90b5b1d4aec32b3a5d784b754f91c1d635dcd04309cfe080048c0f90a"
Jul 17 11:26:21.593850 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:26:21.593652    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-6x65d_odh-ai-gateway-infra(86fb5438-8846-4d53-9010-6428f426f495)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d" podUID="86fb5438-8846-4d53-9010-6428f426f495"
Jul 17 11:26:27.595387 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:27.595351    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"]
Jul 17 11:26:27.595816 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:27.595546    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy" containerID="cri-o://e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44" gracePeriod=30
Jul 17 11:26:29.464760 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:29.464727    2566 patch_prober.go:28] interesting pod/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 11:26:29.465165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:29.464810    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 11:26:30.391366 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:30.391332    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:26:32.851558 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.851535    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:26:32.945294 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945262    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-certs\") pod \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") "
Jul 17 11:26:32.945457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945321    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-token\") pod \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") "
Jul 17 11:26:32.945457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945357    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istiod-ca-cert\") pod \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") "
Jul 17 11:26:32.945457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945395    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-data\") pod \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") "
Jul 17 11:26:32.945457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945430    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-credential-socket\") pod \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") "
Jul 17 11:26:32.945723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945465    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-envoy\") pod \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") "
Jul 17 11:26:32.945723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945510    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5shg\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-kube-api-access-n5shg\") pod \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") "
Jul 17 11:26:32.945723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945535    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-socket\") pod \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") "
Jul 17 11:26:32.945723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945575    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-podinfo\") pod \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\" (UID: \"8e517352-b770-41c8-b0d7-f10dbd5aa71d\") "
Jul 17 11:26:32.945723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945643    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8e517352-b770-41c8-b0d7-f10dbd5aa71d" (UID: "8e517352-b770-41c8-b0d7-f10dbd5aa71d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:26:32.946003 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.945887    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:32.946353 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.946253    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8e517352-b770-41c8-b0d7-f10dbd5aa71d" (UID: "8e517352-b770-41c8-b0d7-f10dbd5aa71d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:26:32.946479 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.946438    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8e517352-b770-41c8-b0d7-f10dbd5aa71d" (UID: "8e517352-b770-41c8-b0d7-f10dbd5aa71d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:26:32.947299 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.947267    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-data" (OuterVolumeSpecName: "istio-data") pod "8e517352-b770-41c8-b0d7-f10dbd5aa71d" (UID: "8e517352-b770-41c8-b0d7-f10dbd5aa71d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:26:32.947425 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.947407    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8e517352-b770-41c8-b0d7-f10dbd5aa71d" (UID: "8e517352-b770-41c8-b0d7-f10dbd5aa71d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:26:32.948255 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.948218    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8e517352-b770-41c8-b0d7-f10dbd5aa71d" (UID: "8e517352-b770-41c8-b0d7-f10dbd5aa71d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:26:32.948364 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.948314    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-kube-api-access-n5shg" (OuterVolumeSpecName: "kube-api-access-n5shg") pod "8e517352-b770-41c8-b0d7-f10dbd5aa71d" (UID: "8e517352-b770-41c8-b0d7-f10dbd5aa71d"). InnerVolumeSpecName "kube-api-access-n5shg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:26:32.948364 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.948332    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8e517352-b770-41c8-b0d7-f10dbd5aa71d" (UID: "8e517352-b770-41c8-b0d7-f10dbd5aa71d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:26:32.948593 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:32.948572    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8e517352-b770-41c8-b0d7-f10dbd5aa71d" (UID: "8e517352-b770-41c8-b0d7-f10dbd5aa71d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:26:33.046590 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.046559    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:33.046590 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.046585    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:33.046780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.046599    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:33.046780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.046611    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:33.046780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.046619    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:33.046780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.046627    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:33.046780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.046636    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5shg\" (UniqueName: \"kubernetes.io/projected/8e517352-b770-41c8-b0d7-f10dbd5aa71d-kube-api-access-n5shg\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:33.046780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.046644    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e517352-b770-41c8-b0d7-f10dbd5aa71d-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:33.650241 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.650203    2566 generic.go:356] "Generic (PLEG): container finished" podID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerID="e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44" exitCode=0
Jul 17 11:26:33.650420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.650247    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" event={"ID":"8e517352-b770-41c8-b0d7-f10dbd5aa71d","Type":"ContainerDied","Data":"e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44"}
Jul 17 11:26:33.650420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.650289    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t" event={"ID":"8e517352-b770-41c8-b0d7-f10dbd5aa71d","Type":"ContainerDied","Data":"9de14b02a78cb8bddc77b09a91ffd00bfd0bf9a38c8e1856a2a7de32755cbf19"}
Jul 17 11:26:33.650420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.650311    2566 scope.go:117] "RemoveContainer" containerID="e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44"
Jul 17 11:26:33.650420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.650247    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"
Jul 17 11:26:33.660323 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.660304    2566 scope.go:117] "RemoveContainer" containerID="b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361"
Jul 17 11:26:33.667862 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.667845    2566 scope.go:117] "RemoveContainer" containerID="e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44"
Jul 17 11:26:33.668075 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:26:33.668056    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44\": container with ID starting with e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44 not found: ID does not exist" containerID="e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44"
Jul 17 11:26:33.668147 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.668086    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44"} err="failed to get container status \"e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44\": rpc error: code = NotFound desc = could not find container \"e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44\": container with ID starting with e3c2753fa75253aa3d5bd86102b4e63f6f95f709c8b7341a7bc00467b5995b44 not found: ID does not exist"
Jul 17 11:26:33.668147 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.668110    2566 scope.go:117] "RemoveContainer" containerID="b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361"
Jul 17 11:26:33.668368 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:26:33.668350    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361\": container with ID starting with b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361 not found: ID does not exist" containerID="b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361"
Jul 17 11:26:33.668435 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.668372    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361"} err="failed to get container status \"b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361\": rpc error: code = NotFound desc = could not find container \"b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361\": container with ID starting with b537f05b3997724230752d451319f357d2a284c8ee1678b77ac647b211ca4361 not found: ID does not exist"
Jul 17 11:26:33.689057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.689036    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"]
Jul 17 11:26:33.692530 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:33.692510    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-829f05b9-openshift-default-658985dd69-hh85t"]
Jul 17 11:26:35.661429 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:35.661396    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" path="/var/lib/kubelet/pods/8e517352-b770-41c8-b0d7-f10dbd5aa71d/volumes"
Jul 17 11:26:36.652325 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:36.652290    2566 scope.go:117] "RemoveContainer" containerID="ba8d9ff90b5b1d4aec32b3a5d784b754f91c1d635dcd04309cfe080048c0f90a"
Jul 17 11:26:37.671914 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:37.671883    2566 generic.go:356] "Generic (PLEG): container finished" podID="86fb5438-8846-4d53-9010-6428f426f495" containerID="a7c27d5fab862540a12b01b511c5c5035ab23cce9ca0e0e03e45bcebb1f8166f" exitCode=6
Jul 17 11:26:37.672308 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:37.671959    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d" event={"ID":"86fb5438-8846-4d53-9010-6428f426f495","Type":"ContainerDied","Data":"a7c27d5fab862540a12b01b511c5c5035ab23cce9ca0e0e03e45bcebb1f8166f"}
Jul 17 11:26:37.672308 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:37.672022    2566 scope.go:117] "RemoveContainer" containerID="ba8d9ff90b5b1d4aec32b3a5d784b754f91c1d635dcd04309cfe080048c0f90a"
Jul 17 11:26:37.672308 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:37.672279    2566 scope.go:117] "RemoveContainer" containerID="a7c27d5fab862540a12b01b511c5c5035ab23cce9ca0e0e03e45bcebb1f8166f"
Jul 17 11:26:37.672480 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:26:37.672457    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-6x65d_odh-ai-gateway-infra(86fb5438-8846-4d53-9010-6428f426f495)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d" podUID="86fb5438-8846-4d53-9010-6428f426f495"
Jul 17 11:26:38.725112 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:38.725078    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"]
Jul 17 11:26:38.860761 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:38.860739    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"
Jul 17 11:26:38.999272 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:38.999174    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86fb5438-8846-4d53-9010-6428f426f495-maas-api-service-ca\") pod \"86fb5438-8846-4d53-9010-6428f426f495\" (UID: \"86fb5438-8846-4d53-9010-6428f426f495\") "
Jul 17 11:26:38.999542 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:38.999520    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86fb5438-8846-4d53-9010-6428f426f495-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "86fb5438-8846-4d53-9010-6428f426f495" (UID: "86fb5438-8846-4d53-9010-6428f426f495"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:26:39.100794 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:39.100763    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86fb5438-8846-4d53-9010-6428f426f495-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:26:39.684576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:39.684550    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"
Jul 17 11:26:39.684735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:39.684590    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d" event={"ID":"86fb5438-8846-4d53-9010-6428f426f495","Type":"ContainerDied","Data":"12747eb179559f1e765bc9c49a4b3e30d23decab4a489f0b87c210b0f9acf242"}
Jul 17 11:26:39.684735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:39.684649    2566 scope.go:117] "RemoveContainer" containerID="a7c27d5fab862540a12b01b511c5c5035ab23cce9ca0e0e03e45bcebb1f8166f"
Jul 17 11:26:39.704165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:39.704144    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"]
Jul 17 11:26:39.708587 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:39.708568    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-6x65d"]
Jul 17 11:26:41.657883 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.657851    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86fb5438-8846-4d53-9010-6428f426f495" path="/var/lib/kubelet/pods/86fb5438-8846-4d53-9010-6428f426f495/volumes"
Jul 17 11:26:41.787826 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.787781    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"]
Jul 17 11:26:41.788163 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788149    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy"
Jul 17 11:26:41.788163 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788164    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy"
Jul 17 11:26:41.788285 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788173    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerName="revoke-keys"
Jul 17 11:26:41.788285 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788178    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerName="revoke-keys"
Jul 17 11:26:41.788285 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788188    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86fb5438-8846-4d53-9010-6428f426f495" containerName="revoke-keys"
Jul 17 11:26:41.788285 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788194    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86fb5438-8846-4d53-9010-6428f426f495" containerName="revoke-keys"
Jul 17 11:26:41.788285 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788202    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86fb5438-8846-4d53-9010-6428f426f495" containerName="revoke-keys"
Jul 17 11:26:41.788285 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788210    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86fb5438-8846-4d53-9010-6428f426f495" containerName="revoke-keys"
Jul 17 11:26:41.788285 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788282    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d73d59d7-4a9d-4001-bcf7-600d0120341e" containerName="revoke-keys"
Jul 17 11:26:41.788628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788299    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="86fb5438-8846-4d53-9010-6428f426f495" containerName="revoke-keys"
Jul 17 11:26:41.788628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788307    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="86fb5438-8846-4d53-9010-6428f426f495" containerName="revoke-keys"
Jul 17 11:26:41.788628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788315    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy"
Jul 17 11:26:41.788628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788320    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy"
Jul 17 11:26:41.788628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.788327    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="86fb5438-8846-4d53-9010-6428f426f495" containerName="revoke-keys"
Jul 17 11:26:41.792736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.792697    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:41.795668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.795649    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-40b693f1-gw-openshift-default-dockercfg-kwc9v\""
Jul 17 11:26:41.812327 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.812306    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"]
Jul 17 11:26:41.923795 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.923683    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-data\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:41.923795 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.923787    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d972c1aa-213c-470a-affd-f7c442d2ffbe-istiod-ca-cert\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:41.923996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.923822    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-credential-socket\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:41.923996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.923840    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-podinfo\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:41.923996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.923863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-socket\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:41.923996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.923879    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-token\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:41.923996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.923961    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-certs\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:41.923996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.923986    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-envoy\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:41.924241 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:41.924018    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-247lc\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-kube-api-access-247lc\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.024949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.024915    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-data\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025118 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.024968    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d972c1aa-213c-470a-affd-f7c442d2ffbe-istiod-ca-cert\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025118 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.024998    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-credential-socket\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025118 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025016    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-podinfo\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025118 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-socket\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025118 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025053    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-token\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025389 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-certs\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025389 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025302    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-envoy\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025389 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025349    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-247lc\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-kube-api-access-247lc\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025389 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025364    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-data\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025579 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025424    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-socket\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025579 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025457    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-credential-socket\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025661 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025611    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-certs\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.025735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.025690    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d972c1aa-213c-470a-affd-f7c442d2ffbe-istiod-ca-cert\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.027417 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.027395    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-envoy\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.027587 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.027569    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-podinfo\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.036469 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.036444    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-token\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.036858 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.036838    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-247lc\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-kube-api-access-247lc\") pod \"e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") " pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.103309 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.103287    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:42.252919 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.252897    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"]
Jul 17 11:26:42.254994 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:26:42.254968    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd972c1aa_213c_470a_affd_f7c442d2ffbe.slice/crio-4fe93efda5676295a8cd280754476fe0a4403c26e1804809f45bbf8b08aa8663 WatchSource:0}: Error finding container 4fe93efda5676295a8cd280754476fe0a4403c26e1804809f45bbf8b08aa8663: Status 404 returned error can't find the container with id 4fe93efda5676295a8cd280754476fe0a4403c26e1804809f45bbf8b08aa8663
Jul 17 11:26:42.257020 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.256983    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:26:42.257072 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.257061    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:26:42.257111 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.257093    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:26:42.702897 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.702856    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" event={"ID":"d972c1aa-213c-470a-affd-f7c442d2ffbe","Type":"ContainerStarted","Data":"da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e"}
Jul 17 11:26:42.703379 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.702913    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" event={"ID":"d972c1aa-213c-470a-affd-f7c442d2ffbe","Type":"ContainerStarted","Data":"4fe93efda5676295a8cd280754476fe0a4403c26e1804809f45bbf8b08aa8663"}
Jul 17 11:26:42.728572 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:42.728519    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" podStartSLOduration=1.7285013230000001 podStartE2EDuration="1.728501323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:26:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:26:42.727677311 +0000 UTC m=+1785.584761253" watchObservedRunningTime="2026-07-17 11:26:42.728501323 +0000 UTC m=+1785.585585270"
Jul 17 11:26:43.104087 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:43.104057    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:43.109070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:43.109048    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:43.708020 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:43.707991    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:43.708986 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:43.708962    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:26:47.687339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.687307    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"]
Jul 17 11:26:47.687732 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.687644    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86fb5438-8846-4d53-9010-6428f426f495" containerName="revoke-keys"
Jul 17 11:26:47.687732 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.687654    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86fb5438-8846-4d53-9010-6428f426f495" containerName="revoke-keys"
Jul 17 11:26:47.687732 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.687681    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy"
Jul 17 11:26:47.687732 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.687686    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e517352-b770-41c8-b0d7-f10dbd5aa71d" containerName="istio-proxy"
Jul 17 11:26:47.690915 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.690894    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"
Jul 17 11:26:47.693944 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.693917    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:26:47.694040 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.694017    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:26:47.700928 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.700907    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"]
Jul 17 11:26:47.877002 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.876970    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda0e97d-e663-4d01-90ae-3a259250eb61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd\" (UID: \"dda0e97d-e663-4d01-90ae-3a259250eb61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"
Jul 17 11:26:47.978081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.978009    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda0e97d-e663-4d01-90ae-3a259250eb61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd\" (UID: \"dda0e97d-e663-4d01-90ae-3a259250eb61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"
Jul 17 11:26:47.978554 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:47.978536    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda0e97d-e663-4d01-90ae-3a259250eb61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd\" (UID: \"dda0e97d-e663-4d01-90ae-3a259250eb61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"
Jul 17 11:26:48.002732 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:48.002694    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"
Jul 17 11:26:48.142000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:48.141978    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"]
Jul 17 11:26:48.144639 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:26:48.144608    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda0e97d_e663_4d01_90ae_3a259250eb61.slice/crio-ac5d8e94349536e627aa2f33dffabc35773690afc9bf6effac386238880dfb5b WatchSource:0}: Error finding container ac5d8e94349536e627aa2f33dffabc35773690afc9bf6effac386238880dfb5b: Status 404 returned error can't find the container with id ac5d8e94349536e627aa2f33dffabc35773690afc9bf6effac386238880dfb5b
Jul 17 11:26:48.736301 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:48.736267    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" event={"ID":"dda0e97d-e663-4d01-90ae-3a259250eb61","Type":"ContainerStarted","Data":"7d3c4e1e0ab1d450292e70f5bada4f90a89e72ecd720e094eb5e66eb90b84701"}
Jul 17 11:26:48.736604 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:48.736331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" event={"ID":"dda0e97d-e663-4d01-90ae-3a259250eb61","Type":"ContainerStarted","Data":"ac5d8e94349536e627aa2f33dffabc35773690afc9bf6effac386238880dfb5b"}
Jul 17 11:26:48.754602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:48.754555    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" podStartSLOduration=1.754540305 podStartE2EDuration="1.754540305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:26:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:26:48.753238546 +0000 UTC m=+1791.610322492" watchObservedRunningTime="2026-07-17 11:26:48.754540305 +0000 UTC m=+1791.611624249"
Jul 17 11:26:49.742568 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:49.742538    2566 generic.go:356] "Generic (PLEG): container finished" podID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerID="7d3c4e1e0ab1d450292e70f5bada4f90a89e72ecd720e094eb5e66eb90b84701" exitCode=7
Jul 17 11:26:49.743025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:49.742606    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" event={"ID":"dda0e97d-e663-4d01-90ae-3a259250eb61","Type":"ContainerDied","Data":"7d3c4e1e0ab1d450292e70f5bada4f90a89e72ecd720e094eb5e66eb90b84701"}
Jul 17 11:26:49.743025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:49.742805    2566 scope.go:117] "RemoveContainer" containerID="7d3c4e1e0ab1d450292e70f5bada4f90a89e72ecd720e094eb5e66eb90b84701"
Jul 17 11:26:50.449258 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:50.449221    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:26:50.748719 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:50.748618    2566 generic.go:356] "Generic (PLEG): container finished" podID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerID="ba5b0390a7a7a2c10662f62be0042b007170fad25b8a071bdae8966e1270b11e" exitCode=7
Jul 17 11:26:50.748719 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:50.748696    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" event={"ID":"dda0e97d-e663-4d01-90ae-3a259250eb61","Type":"ContainerDied","Data":"ba5b0390a7a7a2c10662f62be0042b007170fad25b8a071bdae8966e1270b11e"}
Jul 17 11:26:50.749239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:50.748760    2566 scope.go:117] "RemoveContainer" containerID="7d3c4e1e0ab1d450292e70f5bada4f90a89e72ecd720e094eb5e66eb90b84701"
Jul 17 11:26:50.749239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:50.748943    2566 scope.go:117] "RemoveContainer" containerID="ba5b0390a7a7a2c10662f62be0042b007170fad25b8a071bdae8966e1270b11e"
Jul 17 11:26:50.749239 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:26:50.749193    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd_odh-ai-gateway-infra(dda0e97d-e663-4d01-90ae-3a259250eb61)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61"
Jul 17 11:26:51.756914 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:51.756888    2566 scope.go:117] "RemoveContainer" containerID="ba5b0390a7a7a2c10662f62be0042b007170fad25b8a071bdae8966e1270b11e"
Jul 17 11:26:51.757317 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:26:51.757097    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd_odh-ai-gateway-infra(dda0e97d-e663-4d01-90ae-3a259250eb61)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61"
Jul 17 11:26:57.752816 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:57.752775    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:26:57.756177 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:57.756152    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:26:57.757735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:57.757694    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:26:57.761172 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:57.761155    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:26:58.709934 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:58.709897    2566 patch_prober.go:28] interesting pod/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 17 11:26:58.710153 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:58.709954    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 17 11:26:58.794133 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:58.794099    2566 generic.go:356] "Generic (PLEG): container finished" podID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerID="da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e" exitCode=0
Jul 17 11:26:58.794552 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:58.794168    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" event={"ID":"d972c1aa-213c-470a-affd-f7c442d2ffbe","Type":"ContainerDied","Data":"da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e"}
Jul 17 11:26:58.794603 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:58.794580    2566 scope.go:117] "RemoveContainer" containerID="da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e"
Jul 17 11:26:58.795587 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:58.795556    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:26:58.795696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:58.795640    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:26:58.795696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:58.795688    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:26:59.800860 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:26:59.800826    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" event={"ID":"d972c1aa-213c-470a-affd-f7c442d2ffbe","Type":"ContainerStarted","Data":"4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850"}
Jul 17 11:27:00.103808 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:00.103776    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:27:00.104065 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:00.104045    2566 patch_prober.go:28] interesting pod/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 17 11:27:00.104113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:00.104089    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 17 11:27:01.103993 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:01.103958    2566 patch_prober.go:28] interesting pod/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 17 11:27:01.104382 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:01.104018    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 17 11:27:02.103727 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:02.103668    2566 patch_prober.go:28] interesting pod/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 17 11:27:02.103923 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:02.103757    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 17 11:27:03.103728 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:03.103670    2566 patch_prober.go:28] interesting pod/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 17 11:27:03.104104 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:03.103738    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 17 11:27:04.107878 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:04.107850    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:27:04.108281 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:04.108085    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:27:04.108803 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:04.108788    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:27:06.652081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:06.652044    2566 scope.go:117] "RemoveContainer" containerID="ba5b0390a7a7a2c10662f62be0042b007170fad25b8a071bdae8966e1270b11e"
Jul 17 11:27:07.840148 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:07.840116    2566 generic.go:356] "Generic (PLEG): container finished" podID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerID="2e28a6b5d0db41383dc4440d908ea6794f7540e3bfdf50a0feafd09edc9bb91a" exitCode=7
Jul 17 11:27:07.840148 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:07.840152    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" event={"ID":"dda0e97d-e663-4d01-90ae-3a259250eb61","Type":"ContainerDied","Data":"2e28a6b5d0db41383dc4440d908ea6794f7540e3bfdf50a0feafd09edc9bb91a"}
Jul 17 11:27:07.840622 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:07.840199    2566 scope.go:117] "RemoveContainer" containerID="ba5b0390a7a7a2c10662f62be0042b007170fad25b8a071bdae8966e1270b11e"
Jul 17 11:27:07.840622 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:07.840441    2566 scope.go:117] "RemoveContainer" containerID="2e28a6b5d0db41383dc4440d908ea6794f7540e3bfdf50a0feafd09edc9bb91a"
Jul 17 11:27:07.840728 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:27:07.840648    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd_odh-ai-gateway-infra(dda0e97d-e663-4d01-90ae-3a259250eb61)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61"
Jul 17 11:27:08.870756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:08.870669    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"]
Jul 17 11:27:09.006991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:09.006964    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"
Jul 17 11:27:09.059291 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:09.059263    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda0e97d-e663-4d01-90ae-3a259250eb61-maas-api-service-ca\") pod \"dda0e97d-e663-4d01-90ae-3a259250eb61\" (UID: \"dda0e97d-e663-4d01-90ae-3a259250eb61\") "
Jul 17 11:27:09.059560 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:09.059538    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda0e97d-e663-4d01-90ae-3a259250eb61-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dda0e97d-e663-4d01-90ae-3a259250eb61" (UID: "dda0e97d-e663-4d01-90ae-3a259250eb61"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:27:09.160189 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:09.160113    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda0e97d-e663-4d01-90ae-3a259250eb61-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:09.852451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:09.852416    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd" event={"ID":"dda0e97d-e663-4d01-90ae-3a259250eb61","Type":"ContainerDied","Data":"ac5d8e94349536e627aa2f33dffabc35773690afc9bf6effac386238880dfb5b"}
Jul 17 11:27:09.852451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:09.852437    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"
Jul 17 11:27:09.852683 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:09.852483    2566 scope.go:117] "RemoveContainer" containerID="2e28a6b5d0db41383dc4440d908ea6794f7540e3bfdf50a0feafd09edc9bb91a"
Jul 17 11:27:09.873497 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:09.873476    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"]
Jul 17 11:27:09.880498 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:09.880478    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-g2tmd"]
Jul 17 11:27:11.658474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:11.658438    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" path="/var/lib/kubelet/pods/dda0e97d-e663-4d01-90ae-3a259250eb61/volumes"
Jul 17 11:27:17.735505 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.735465    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"]
Jul 17 11:27:17.736036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.736012    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerName="revoke-keys"
Jul 17 11:27:17.736116 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.736040    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerName="revoke-keys"
Jul 17 11:27:17.736116 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.736082    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerName="revoke-keys"
Jul 17 11:27:17.736116 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.736093    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerName="revoke-keys"
Jul 17 11:27:17.736265 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.736182    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerName="revoke-keys"
Jul 17 11:27:17.736265 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.736195    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerName="revoke-keys"
Jul 17 11:27:17.740164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.740141    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"
Jul 17 11:27:17.743160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.743137    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:27:17.743235 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.743178    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:27:17.748958 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.748903    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"]
Jul 17 11:27:17.834574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.834550    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70298a38-b3f5-4324-bad2-8ced3630e815-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-2n24l\" (UID: \"70298a38-b3f5-4324-bad2-8ced3630e815\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"
Jul 17 11:27:17.935735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.935688    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70298a38-b3f5-4324-bad2-8ced3630e815-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-2n24l\" (UID: \"70298a38-b3f5-4324-bad2-8ced3630e815\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"
Jul 17 11:27:17.936268 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:17.936250    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70298a38-b3f5-4324-bad2-8ced3630e815-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-2n24l\" (UID: \"70298a38-b3f5-4324-bad2-8ced3630e815\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"
Jul 17 11:27:18.049792 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:18.049733    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"
Jul 17 11:27:18.191158 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:18.191136    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"]
Jul 17 11:27:18.193354 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:27:18.193320    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70298a38_b3f5_4324_bad2_8ced3630e815.slice/crio-92f37ffda128534cc3656041d00bd153bd843a8a4be8ac0f826387d0172a7776 WatchSource:0}: Error finding container 92f37ffda128534cc3656041d00bd153bd843a8a4be8ac0f826387d0172a7776: Status 404 returned error can't find the container with id 92f37ffda128534cc3656041d00bd153bd843a8a4be8ac0f826387d0172a7776
Jul 17 11:27:18.898846 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:18.898816    2566 generic.go:356] "Generic (PLEG): container finished" podID="70298a38-b3f5-4324-bad2-8ced3630e815" containerID="92dc8e0be23ce1045df4dccf0f9874674d312e8e81e3e9972790ed6413dd3e0e" exitCode=6
Jul 17 11:27:18.899174 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:18.898901    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" event={"ID":"70298a38-b3f5-4324-bad2-8ced3630e815","Type":"ContainerDied","Data":"92dc8e0be23ce1045df4dccf0f9874674d312e8e81e3e9972790ed6413dd3e0e"}
Jul 17 11:27:18.899174 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:18.898952    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" event={"ID":"70298a38-b3f5-4324-bad2-8ced3630e815","Type":"ContainerStarted","Data":"92f37ffda128534cc3656041d00bd153bd843a8a4be8ac0f826387d0172a7776"}
Jul 17 11:27:18.899174 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:18.899088    2566 scope.go:117] "RemoveContainer" containerID="92dc8e0be23ce1045df4dccf0f9874674d312e8e81e3e9972790ed6413dd3e0e"
Jul 17 11:27:19.910500 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:19.910472    2566 generic.go:356] "Generic (PLEG): container finished" podID="70298a38-b3f5-4324-bad2-8ced3630e815" containerID="8461af64aac0743c98ee256b81c5daa1c8a0add47049458ee25729f5a5da06f1" exitCode=6
Jul 17 11:27:19.911027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:19.910554    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" event={"ID":"70298a38-b3f5-4324-bad2-8ced3630e815","Type":"ContainerDied","Data":"8461af64aac0743c98ee256b81c5daa1c8a0add47049458ee25729f5a5da06f1"}
Jul 17 11:27:19.911027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:19.910645    2566 scope.go:117] "RemoveContainer" containerID="92dc8e0be23ce1045df4dccf0f9874674d312e8e81e3e9972790ed6413dd3e0e"
Jul 17 11:27:19.911027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:19.910853    2566 scope.go:117] "RemoveContainer" containerID="8461af64aac0743c98ee256b81c5daa1c8a0add47049458ee25729f5a5da06f1"
Jul 17 11:27:19.911183 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:27:19.911084    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-2n24l_odh-ai-gateway-infra(70298a38-b3f5-4324-bad2-8ced3630e815)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" podUID="70298a38-b3f5-4324-bad2-8ced3630e815"
Jul 17 11:27:20.917602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:20.917572    2566 scope.go:117] "RemoveContainer" containerID="8461af64aac0743c98ee256b81c5daa1c8a0add47049458ee25729f5a5da06f1"
Jul 17 11:27:20.918077 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:27:20.917768    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-2n24l_odh-ai-gateway-infra(70298a38-b3f5-4324-bad2-8ced3630e815)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" podUID="70298a38-b3f5-4324-bad2-8ced3630e815"
Jul 17 11:27:33.652780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:33.652742    2566 scope.go:117] "RemoveContainer" containerID="8461af64aac0743c98ee256b81c5daa1c8a0add47049458ee25729f5a5da06f1"
Jul 17 11:27:33.983516 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:33.983472    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" event={"ID":"70298a38-b3f5-4324-bad2-8ced3630e815","Type":"ContainerStarted","Data":"744b3475fe013defb4d003d861c3fb45d27d3cc4c0c0bce8c180c5c69f1c14ba"}
Jul 17 11:27:34.000861 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:34.000810    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" podStartSLOduration=17.000794762 podStartE2EDuration="17.000794762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:27:33.999959994 +0000 UTC m=+1836.857043940" watchObservedRunningTime="2026-07-17 11:27:34.000794762 +0000 UTC m=+1836.857878718"
Jul 17 11:27:34.989876 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:34.989841    2566 generic.go:356] "Generic (PLEG): container finished" podID="70298a38-b3f5-4324-bad2-8ced3630e815" containerID="744b3475fe013defb4d003d861c3fb45d27d3cc4c0c0bce8c180c5c69f1c14ba" exitCode=6
Jul 17 11:27:34.990342 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:34.989900    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" event={"ID":"70298a38-b3f5-4324-bad2-8ced3630e815","Type":"ContainerDied","Data":"744b3475fe013defb4d003d861c3fb45d27d3cc4c0c0bce8c180c5c69f1c14ba"}
Jul 17 11:27:34.990342 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:34.989958    2566 scope.go:117] "RemoveContainer" containerID="8461af64aac0743c98ee256b81c5daa1c8a0add47049458ee25729f5a5da06f1"
Jul 17 11:27:34.990342 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:34.990205    2566 scope.go:117] "RemoveContainer" containerID="744b3475fe013defb4d003d861c3fb45d27d3cc4c0c0bce8c180c5c69f1c14ba"
Jul 17 11:27:34.990475 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:27:34.990404    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-2n24l_odh-ai-gateway-infra(70298a38-b3f5-4324-bad2-8ced3630e815)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" podUID="70298a38-b3f5-4324-bad2-8ced3630e815"
Jul 17 11:27:35.016525 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:35.016497    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"]
Jul 17 11:27:36.127221 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:36.127193    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"
Jul 17 11:27:36.302806 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:36.302700    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70298a38-b3f5-4324-bad2-8ced3630e815-maas-api-service-ca\") pod \"70298a38-b3f5-4324-bad2-8ced3630e815\" (UID: \"70298a38-b3f5-4324-bad2-8ced3630e815\") "
Jul 17 11:27:36.303062 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:36.303038    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70298a38-b3f5-4324-bad2-8ced3630e815-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "70298a38-b3f5-4324-bad2-8ced3630e815" (UID: "70298a38-b3f5-4324-bad2-8ced3630e815"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:27:36.403559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:36.403527    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70298a38-b3f5-4324-bad2-8ced3630e815-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:37.001943 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:37.001914    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"
Jul 17 11:27:37.002121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:37.001913    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l" event={"ID":"70298a38-b3f5-4324-bad2-8ced3630e815","Type":"ContainerDied","Data":"92f37ffda128534cc3656041d00bd153bd843a8a4be8ac0f826387d0172a7776"}
Jul 17 11:27:37.002121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:37.002053    2566 scope.go:117] "RemoveContainer" containerID="744b3475fe013defb4d003d861c3fb45d27d3cc4c0c0bce8c180c5c69f1c14ba"
Jul 17 11:27:37.024743 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:37.024721    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"]
Jul 17 11:27:37.030471 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:37.030447    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-2n24l"]
Jul 17 11:27:37.663055 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:37.663014    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" path="/var/lib/kubelet/pods/70298a38-b3f5-4324-bad2-8ced3630e815/volumes"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.744608    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"]
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745614    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerName="revoke-keys"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745634    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerName="revoke-keys"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745671    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" containerName="revoke-keys"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745679    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" containerName="revoke-keys"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745732    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" containerName="revoke-keys"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745741    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" containerName="revoke-keys"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745910    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" containerName="revoke-keys"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745928    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" containerName="revoke-keys"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745937    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" containerName="revoke-keys"
Jul 17 11:27:47.747633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.745944    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda0e97d-e663-4d01-90ae-3a259250eb61" containerName="revoke-keys"
Jul 17 11:27:47.751502 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.751480    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"
Jul 17 11:27:47.754544 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.754527    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:27:47.754911 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.754888    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:27:47.758239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.758215    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"]
Jul 17 11:27:47.799013 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.798981    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14db1518-cc25-4350-9e75-eead39c72232-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8\" (UID: \"14db1518-cc25-4350-9e75-eead39c72232\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"
Jul 17 11:27:47.821075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.821053    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"]
Jul 17 11:27:47.821284 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.821253    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy" containerID="cri-o://4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850" gracePeriod=30
Jul 17 11:27:47.900028 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.900002    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14db1518-cc25-4350-9e75-eead39c72232-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8\" (UID: \"14db1518-cc25-4350-9e75-eead39c72232\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"
Jul 17 11:27:47.900639 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:47.900618    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14db1518-cc25-4350-9e75-eead39c72232-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8\" (UID: \"14db1518-cc25-4350-9e75-eead39c72232\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"
Jul 17 11:27:48.062956 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:48.062879    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"
Jul 17 11:27:48.205062 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:48.205031    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"]
Jul 17 11:27:48.208082 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:27:48.208056    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14db1518_cc25_4350_9e75_eead39c72232.slice/crio-136df3e1ad14b754161ab37475a197fa7963b9b03277729e8de2c4c5ee500969 WatchSource:0}: Error finding container 136df3e1ad14b754161ab37475a197fa7963b9b03277729e8de2c4c5ee500969: Status 404 returned error can't find the container with id 136df3e1ad14b754161ab37475a197fa7963b9b03277729e8de2c4c5ee500969
Jul 17 11:27:49.060112 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:49.060079    2566 generic.go:356] "Generic (PLEG): container finished" podID="14db1518-cc25-4350-9e75-eead39c72232" containerID="b4404d9b111a2ff6d80f415246085698af4910784879d0c153bfc564783fbd05" exitCode=7
Jul 17 11:27:49.060434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:49.060152    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" event={"ID":"14db1518-cc25-4350-9e75-eead39c72232","Type":"ContainerDied","Data":"b4404d9b111a2ff6d80f415246085698af4910784879d0c153bfc564783fbd05"}
Jul 17 11:27:49.060434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:49.060186    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" event={"ID":"14db1518-cc25-4350-9e75-eead39c72232","Type":"ContainerStarted","Data":"136df3e1ad14b754161ab37475a197fa7963b9b03277729e8de2c4c5ee500969"}
Jul 17 11:27:49.060434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:49.060317    2566 scope.go:117] "RemoveContainer" containerID="b4404d9b111a2ff6d80f415246085698af4910784879d0c153bfc564783fbd05"
Jul 17 11:27:49.110462 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:49.110431    2566 patch_prober.go:28] interesting pod/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 11:27:49.110557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:49.110479    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 11:27:50.067314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:50.067237    2566 generic.go:356] "Generic (PLEG): container finished" podID="14db1518-cc25-4350-9e75-eead39c72232" containerID="56c6239791917fabd86cc46ce32319c6a79ebe72278b6ec3fc3724c5de499903" exitCode=7
Jul 17 11:27:50.067314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:50.067306    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" event={"ID":"14db1518-cc25-4350-9e75-eead39c72232","Type":"ContainerDied","Data":"56c6239791917fabd86cc46ce32319c6a79ebe72278b6ec3fc3724c5de499903"}
Jul 17 11:27:50.067771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:50.067349    2566 scope.go:117] "RemoveContainer" containerID="b4404d9b111a2ff6d80f415246085698af4910784879d0c153bfc564783fbd05"
Jul 17 11:27:50.067771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:50.067504    2566 scope.go:117] "RemoveContainer" containerID="56c6239791917fabd86cc46ce32319c6a79ebe72278b6ec3fc3724c5de499903"
Jul 17 11:27:50.067845 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:27:50.067761    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8_odh-ai-gateway-infra(14db1518-cc25-4350-9e75-eead39c72232)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" podUID="14db1518-cc25-4350-9e75-eead39c72232"
Jul 17 11:27:50.699373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:50.699340    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:27:51.074146 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:51.074123    2566 scope.go:117] "RemoveContainer" containerID="56c6239791917fabd86cc46ce32319c6a79ebe72278b6ec3fc3724c5de499903"
Jul 17 11:27:51.074540 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:27:51.074314    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8_odh-ai-gateway-infra(14db1518-cc25-4350-9e75-eead39c72232)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" podUID="14db1518-cc25-4350-9e75-eead39c72232"
Jul 17 11:27:53.084125 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.084051    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:27:53.086229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.086201    2566 generic.go:356] "Generic (PLEG): container finished" podID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerID="4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850" exitCode=0
Jul 17 11:27:53.086345 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.086242    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"
Jul 17 11:27:53.086345 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.086272    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" event={"ID":"d972c1aa-213c-470a-affd-f7c442d2ffbe","Type":"ContainerDied","Data":"4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850"}
Jul 17 11:27:53.086422 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.086346    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp" event={"ID":"d972c1aa-213c-470a-affd-f7c442d2ffbe","Type":"ContainerDied","Data":"4fe93efda5676295a8cd280754476fe0a4403c26e1804809f45bbf8b08aa8663"}
Jul 17 11:27:53.086422 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.086358    2566 scope.go:117] "RemoveContainer" containerID="4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850"
Jul 17 11:27:53.095024 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.095005    2566 scope.go:117] "RemoveContainer" containerID="da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e"
Jul 17 11:27:53.104350 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.104329    2566 scope.go:117] "RemoveContainer" containerID="4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850"
Jul 17 11:27:53.104639 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:27:53.104612    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850\": container with ID starting with 4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850 not found: ID does not exist" containerID="4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850"
Jul 17 11:27:53.104700 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.104649    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850"} err="failed to get container status \"4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850\": rpc error: code = NotFound desc = could not find container \"4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850\": container with ID starting with 4a325e6ba9a92aa75f83e2a7dfcc017d3640cc229defac0b926712c85edc1850 not found: ID does not exist"
Jul 17 11:27:53.104700 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.104669    2566 scope.go:117] "RemoveContainer" containerID="da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e"
Jul 17 11:27:53.104950 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:27:53.104920    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e\": container with ID starting with da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e not found: ID does not exist" containerID="da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e"
Jul 17 11:27:53.105059 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.104953    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e"} err="failed to get container status \"da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e\": rpc error: code = NotFound desc = could not find container \"da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e\": container with ID starting with da3ddcc9a83f651c4039ecb23337333d4c3fad2c8e213e820421727eabc5862e not found: ID does not exist"
Jul 17 11:27:53.145147 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145107    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-podinfo\") pod \"d972c1aa-213c-470a-affd-f7c442d2ffbe\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") "
Jul 17 11:27:53.145286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145163    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-envoy\") pod \"d972c1aa-213c-470a-affd-f7c442d2ffbe\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") "
Jul 17 11:27:53.145286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145200    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d972c1aa-213c-470a-affd-f7c442d2ffbe-istiod-ca-cert\") pod \"d972c1aa-213c-470a-affd-f7c442d2ffbe\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") "
Jul 17 11:27:53.145286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145241    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-token\") pod \"d972c1aa-213c-470a-affd-f7c442d2ffbe\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") "
Jul 17 11:27:53.145414 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145287    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-247lc\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-kube-api-access-247lc\") pod \"d972c1aa-213c-470a-affd-f7c442d2ffbe\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") "
Jul 17 11:27:53.145414 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145356    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-certs\") pod \"d972c1aa-213c-470a-affd-f7c442d2ffbe\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") "
Jul 17 11:27:53.145414 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145405    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-socket\") pod \"d972c1aa-213c-470a-affd-f7c442d2ffbe\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") "
Jul 17 11:27:53.145563 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145473    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-data\") pod \"d972c1aa-213c-470a-affd-f7c442d2ffbe\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") "
Jul 17 11:27:53.145563 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145501    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-credential-socket\") pod \"d972c1aa-213c-470a-affd-f7c442d2ffbe\" (UID: \"d972c1aa-213c-470a-affd-f7c442d2ffbe\") "
Jul 17 11:27:53.145668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145629    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d972c1aa-213c-470a-affd-f7c442d2ffbe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d972c1aa-213c-470a-affd-f7c442d2ffbe" (UID: "d972c1aa-213c-470a-affd-f7c442d2ffbe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:27:53.145757 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145662    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d972c1aa-213c-470a-affd-f7c442d2ffbe" (UID: "d972c1aa-213c-470a-affd-f7c442d2ffbe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:27:53.145934 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145910    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d972c1aa-213c-470a-affd-f7c442d2ffbe" (UID: "d972c1aa-213c-470a-affd-f7c442d2ffbe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:27:53.146106 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145932    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d972c1aa-213c-470a-affd-f7c442d2ffbe" (UID: "d972c1aa-213c-470a-affd-f7c442d2ffbe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:27:53.146198 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.145977    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:53.146305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.146292    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d972c1aa-213c-470a-affd-f7c442d2ffbe-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:53.146974 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.146946    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-data" (OuterVolumeSpecName: "istio-data") pod "d972c1aa-213c-470a-affd-f7c442d2ffbe" (UID: "d972c1aa-213c-470a-affd-f7c442d2ffbe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:27:53.147733 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.147677    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d972c1aa-213c-470a-affd-f7c442d2ffbe" (UID: "d972c1aa-213c-470a-affd-f7c442d2ffbe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:27:53.147935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.147908    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-token" (OuterVolumeSpecName: "istio-token") pod "d972c1aa-213c-470a-affd-f7c442d2ffbe" (UID: "d972c1aa-213c-470a-affd-f7c442d2ffbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:27:53.148096 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.148077    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d972c1aa-213c-470a-affd-f7c442d2ffbe" (UID: "d972c1aa-213c-470a-affd-f7c442d2ffbe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:27:53.148318 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.148291    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-kube-api-access-247lc" (OuterVolumeSpecName: "kube-api-access-247lc") pod "d972c1aa-213c-470a-affd-f7c442d2ffbe" (UID: "d972c1aa-213c-470a-affd-f7c442d2ffbe"). InnerVolumeSpecName "kube-api-access-247lc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:27:53.247802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.247770    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:53.247802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.247799    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:53.247802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.247811    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:53.248045 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.247824    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:53.248045 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.247836    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:53.248045 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.247846    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:53.248045 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.247856    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-247lc\" (UniqueName: \"kubernetes.io/projected/d972c1aa-213c-470a-affd-f7c442d2ffbe-kube-api-access-247lc\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:27:53.408900 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.408873    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"]
Jul 17 11:27:53.413852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.413828    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-40b693f1-gw-openshift-default-57455b8dd7-d4jkp"]
Jul 17 11:27:53.658793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:27:53.658692    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" path="/var/lib/kubelet/pods/d972c1aa-213c-470a-affd-f7c442d2ffbe/volumes"
Jul 17 11:28:00.364298 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.364260    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"]
Jul 17 11:28:00.364884 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.364863    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" containerName="revoke-keys"
Jul 17 11:28:00.364956 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.364887    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="70298a38-b3f5-4324-bad2-8ced3630e815" containerName="revoke-keys"
Jul 17 11:28:00.364956 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.364920    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy"
Jul 17 11:28:00.364956 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.364928    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy"
Jul 17 11:28:00.364956 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.364944    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy"
Jul 17 11:28:00.364956 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.364953    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy"
Jul 17 11:28:00.365123 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.365054    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy"
Jul 17 11:28:00.370368 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.370343    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.373333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.373303    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-45989be5-openshift-default-dockercfg-8665c\""
Jul 17 11:28:00.384590 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.384563    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"]
Jul 17 11:28:00.410731 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.410691    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-envoy\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.410866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.410741    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s968j\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-kube-api-access-s968j\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.410866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.410772    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-credential-socket\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.410866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.410792    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-token\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.410866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.410832    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-podinfo\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.411144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.410930    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-certs\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.411144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.411011    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-data\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.411144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.411045    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istiod-ca-cert\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.411144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.411090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-socket\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.512524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.512488    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-token\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.512663 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.512550    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-podinfo\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.512663 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.512590    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-certs\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.512821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.512741    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-data\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.512821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.512778    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istiod-ca-cert\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.512821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.512813    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-socket\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.512972 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.512845    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-envoy\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.512972 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.512870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s968j\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-kube-api-access-s968j\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.512972 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.512911    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-credential-socket\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.513119 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.513045    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-certs\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.513119 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.513103    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-data\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.513225 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.513133    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-socket\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.513300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.513238    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-credential-socket\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.513478 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.513454    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istiod-ca-cert\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.515162 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.515133    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-envoy\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.515393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.515374    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-podinfo\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.533421 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.533395    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-token\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.533610 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.533588    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s968j\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-kube-api-access-s968j\") pod \"e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") " pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.681084 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.681008    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:00.816668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.816639    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"]
Jul 17 11:28:00.818311 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:28:00.818284    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d1a1a8_eeeb_4fdf_a918_57ddd018af4c.slice/crio-1d99df53b6190c68a5c0a2268313af2c79de17cde59256058d0b56424145ae13 WatchSource:0}: Error finding container 1d99df53b6190c68a5c0a2268313af2c79de17cde59256058d0b56424145ae13: Status 404 returned error can't find the container with id 1d99df53b6190c68a5c0a2268313af2c79de17cde59256058d0b56424145ae13
Jul 17 11:28:00.820399 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.820370    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:28:00.820466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.820433    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:28:00.820466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:00.820461    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:28:01.132173 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:01.132135    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" event={"ID":"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c","Type":"ContainerStarted","Data":"f848e79ca95ad41e9aa54e867c80ad000ee50f2b9bba65bcd98fab0b929834cc"}
Jul 17 11:28:01.132367 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:01.132189    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" event={"ID":"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c","Type":"ContainerStarted","Data":"1d99df53b6190c68a5c0a2268313af2c79de17cde59256058d0b56424145ae13"}
Jul 17 11:28:01.157060 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:01.157011    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" podStartSLOduration=1.156996359 podStartE2EDuration="1.156996359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:28:01.155598947 +0000 UTC m=+1864.012682892" watchObservedRunningTime="2026-07-17 11:28:01.156996359 +0000 UTC m=+1864.014080303"
Jul 17 11:28:01.681601 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:01.681573    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:01.686082 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:01.686061    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:02.139612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:02.139582    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:02.143604 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:02.143578    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:03.651470 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:03.651449    2566 scope.go:117] "RemoveContainer" containerID="56c6239791917fabd86cc46ce32319c6a79ebe72278b6ec3fc3724c5de499903"
Jul 17 11:28:04.149354 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:04.149317    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" event={"ID":"14db1518-cc25-4350-9e75-eead39c72232","Type":"ContainerStarted","Data":"ee98d807f29e2841a23343fac3a68272a91a5f706edacc1e7e04d3510b10321e"}
Jul 17 11:28:04.172233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:04.172186    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" podStartSLOduration=17.172173802 podStartE2EDuration="17.172173802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:28:04.170405153 +0000 UTC m=+1867.027489100" watchObservedRunningTime="2026-07-17 11:28:04.172173802 +0000 UTC m=+1867.029257742"
Jul 17 11:28:04.663023 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:04.662948    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:28:05.155443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:05.155411    2566 generic.go:356] "Generic (PLEG): container finished" podID="14db1518-cc25-4350-9e75-eead39c72232" containerID="ee98d807f29e2841a23343fac3a68272a91a5f706edacc1e7e04d3510b10321e" exitCode=6
Jul 17 11:28:05.155586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:05.155459    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" event={"ID":"14db1518-cc25-4350-9e75-eead39c72232","Type":"ContainerDied","Data":"ee98d807f29e2841a23343fac3a68272a91a5f706edacc1e7e04d3510b10321e"}
Jul 17 11:28:05.155586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:05.155507    2566 scope.go:117] "RemoveContainer" containerID="56c6239791917fabd86cc46ce32319c6a79ebe72278b6ec3fc3724c5de499903"
Jul 17 11:28:05.155771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:05.155755    2566 scope.go:117] "RemoveContainer" containerID="ee98d807f29e2841a23343fac3a68272a91a5f706edacc1e7e04d3510b10321e"
Jul 17 11:28:05.155981 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:05.155958    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8_odh-ai-gateway-infra(14db1518-cc25-4350-9e75-eead39c72232)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" podUID="14db1518-cc25-4350-9e75-eead39c72232"
Jul 17 11:28:05.180958 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:05.180936    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"]
Jul 17 11:28:06.324451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.324421    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"
Jul 17 11:28:06.465139 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.465051    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14db1518-cc25-4350-9e75-eead39c72232-maas-api-service-ca\") pod \"14db1518-cc25-4350-9e75-eead39c72232\" (UID: \"14db1518-cc25-4350-9e75-eead39c72232\") "
Jul 17 11:28:06.465470 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.465442    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14db1518-cc25-4350-9e75-eead39c72232-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "14db1518-cc25-4350-9e75-eead39c72232" (UID: "14db1518-cc25-4350-9e75-eead39c72232"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:28:06.566673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.566636    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14db1518-cc25-4350-9e75-eead39c72232-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:28:06.684239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.684199    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"]
Jul 17 11:28:06.684571 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.684559    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14db1518-cc25-4350-9e75-eead39c72232" containerName="revoke-keys"
Jul 17 11:28:06.684616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.684572    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="14db1518-cc25-4350-9e75-eead39c72232" containerName="revoke-keys"
Jul 17 11:28:06.684616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.684598    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14db1518-cc25-4350-9e75-eead39c72232" containerName="revoke-keys"
Jul 17 11:28:06.684616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.684604    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="14db1518-cc25-4350-9e75-eead39c72232" containerName="revoke-keys"
Jul 17 11:28:06.684745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.684656    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="14db1518-cc25-4350-9e75-eead39c72232" containerName="revoke-keys"
Jul 17 11:28:06.684745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.684664    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d972c1aa-213c-470a-affd-f7c442d2ffbe" containerName="istio-proxy"
Jul 17 11:28:06.684745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.684673    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="14db1518-cc25-4350-9e75-eead39c72232" containerName="revoke-keys"
Jul 17 11:28:06.687580 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.687560    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"
Jul 17 11:28:06.696065 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.696044    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"]
Jul 17 11:28:06.768048 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.767952    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e68c051-9514-4695-967a-3d59d19daed8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-znrfl\" (UID: \"1e68c051-9514-4695-967a-3d59d19daed8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"
Jul 17 11:28:06.869098 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.869062    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e68c051-9514-4695-967a-3d59d19daed8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-znrfl\" (UID: \"1e68c051-9514-4695-967a-3d59d19daed8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"
Jul 17 11:28:06.869727 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.869686    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e68c051-9514-4695-967a-3d59d19daed8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-znrfl\" (UID: \"1e68c051-9514-4695-967a-3d59d19daed8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"
Jul 17 11:28:06.999045 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:06.999006    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"
Jul 17 11:28:07.155274 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.155237    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"]
Jul 17 11:28:07.161397 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:28:07.161367    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e68c051_9514_4695_967a_3d59d19daed8.slice/crio-6534d0034b9f01bc97c2cebb31c67941c7ddbfe860592e4732251f6aca0570c9 WatchSource:0}: Error finding container 6534d0034b9f01bc97c2cebb31c67941c7ddbfe860592e4732251f6aca0570c9: Status 404 returned error can't find the container with id 6534d0034b9f01bc97c2cebb31c67941c7ddbfe860592e4732251f6aca0570c9
Jul 17 11:28:07.168171 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.168139    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl" event={"ID":"1e68c051-9514-4695-967a-3d59d19daed8","Type":"ContainerStarted","Data":"6534d0034b9f01bc97c2cebb31c67941c7ddbfe860592e4732251f6aca0570c9"}
Jul 17 11:28:07.169537 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.169517    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8" event={"ID":"14db1518-cc25-4350-9e75-eead39c72232","Type":"ContainerDied","Data":"136df3e1ad14b754161ab37475a197fa7963b9b03277729e8de2c4c5ee500969"}
Jul 17 11:28:07.169634 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.169565    2566 scope.go:117] "RemoveContainer" containerID="ee98d807f29e2841a23343fac3a68272a91a5f706edacc1e7e04d3510b10321e"
Jul 17 11:28:07.169634 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.169562    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"
Jul 17 11:28:07.198890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.195960    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"]
Jul 17 11:28:07.200610 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.200589    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-fbzg8"]
Jul 17 11:28:07.658979 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.658900    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14db1518-cc25-4350-9e75-eead39c72232" path="/var/lib/kubelet/pods/14db1518-cc25-4350-9e75-eead39c72232/volumes"
Jul 17 11:28:07.796984 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.796947    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"]
Jul 17 11:28:07.797336 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.797323    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14db1518-cc25-4350-9e75-eead39c72232" containerName="revoke-keys"
Jul 17 11:28:07.797336 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.797337    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="14db1518-cc25-4350-9e75-eead39c72232" containerName="revoke-keys"
Jul 17 11:28:07.797416 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.797410    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="14db1518-cc25-4350-9e75-eead39c72232" containerName="revoke-keys"
Jul 17 11:28:07.800547 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.800531    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"
Jul 17 11:28:07.811378 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.811354    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"]
Jul 17 11:28:07.879450 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.879410    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e9babf-cd4a-4a28-8afb-382f3c13f391-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx\" (UID: \"22e9babf-cd4a-4a28-8afb-382f3c13f391\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"
Jul 17 11:28:07.980872 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.980783    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e9babf-cd4a-4a28-8afb-382f3c13f391-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx\" (UID: \"22e9babf-cd4a-4a28-8afb-382f3c13f391\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"
Jul 17 11:28:07.981393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:07.981372    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e9babf-cd4a-4a28-8afb-382f3c13f391-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx\" (UID: \"22e9babf-cd4a-4a28-8afb-382f3c13f391\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"
Jul 17 11:28:08.111190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:08.111140    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"
Jul 17 11:28:08.176071 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:08.176028    2566 generic.go:356] "Generic (PLEG): container finished" podID="1e68c051-9514-4695-967a-3d59d19daed8" containerID="1e723cd228ec503954eddff582cb26fa379d28c5f7543cb2e64c96a66c0b6b43" exitCode=7
Jul 17 11:28:08.176285 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:08.176148    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl" event={"ID":"1e68c051-9514-4695-967a-3d59d19daed8","Type":"ContainerDied","Data":"1e723cd228ec503954eddff582cb26fa379d28c5f7543cb2e64c96a66c0b6b43"}
Jul 17 11:28:08.176388 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:08.176371    2566 scope.go:117] "RemoveContainer" containerID="1e723cd228ec503954eddff582cb26fa379d28c5f7543cb2e64c96a66c0b6b43"
Jul 17 11:28:08.322441 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:08.322419    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"]
Jul 17 11:28:09.184039 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:09.184004    2566 generic.go:356] "Generic (PLEG): container finished" podID="1e68c051-9514-4695-967a-3d59d19daed8" containerID="32b53313e01e40a8deca2002e2b751a12375b9af46eedceb114da5257ba213a4" exitCode=7
Jul 17 11:28:09.184456 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:09.184091    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl" event={"ID":"1e68c051-9514-4695-967a-3d59d19daed8","Type":"ContainerDied","Data":"32b53313e01e40a8deca2002e2b751a12375b9af46eedceb114da5257ba213a4"}
Jul 17 11:28:09.184456 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:09.184151    2566 scope.go:117] "RemoveContainer" containerID="1e723cd228ec503954eddff582cb26fa379d28c5f7543cb2e64c96a66c0b6b43"
Jul 17 11:28:09.184456 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:09.184291    2566 scope.go:117] "RemoveContainer" containerID="32b53313e01e40a8deca2002e2b751a12375b9af46eedceb114da5257ba213a4"
Jul 17 11:28:09.184590 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:09.184519    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-znrfl_odh-ai-gateway-infra(1e68c051-9514-4695-967a-3d59d19daed8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl" podUID="1e68c051-9514-4695-967a-3d59d19daed8"
Jul 17 11:28:09.185744 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:09.185695    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" event={"ID":"22e9babf-cd4a-4a28-8afb-382f3c13f391","Type":"ContainerStarted","Data":"ececf67d53902a9b966673ca7f27a15081ae6b1c9e919c8faf1db18a797844a9"}
Jul 17 11:28:09.185863 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:09.185776    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" event={"ID":"22e9babf-cd4a-4a28-8afb-382f3c13f391","Type":"ContainerStarted","Data":"653e19dd2a98c5a0170613642d07cd68525290a847395dc4893ac35746621722"}
Jul 17 11:28:09.220103 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:09.220058    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" podStartSLOduration=2.22004339 podStartE2EDuration="2.22004339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:28:09.218570187 +0000 UTC m=+1872.075654133" watchObservedRunningTime="2026-07-17 11:28:09.22004339 +0000 UTC m=+1872.077127334"
Jul 17 11:28:10.192445 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:10.192413    2566 scope.go:117] "RemoveContainer" containerID="32b53313e01e40a8deca2002e2b751a12375b9af46eedceb114da5257ba213a4"
Jul 17 11:28:10.192908 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:10.192651    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-znrfl_odh-ai-gateway-infra(1e68c051-9514-4695-967a-3d59d19daed8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl" podUID="1e68c051-9514-4695-967a-3d59d19daed8"
Jul 17 11:28:10.193666 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:10.193645    2566 generic.go:356] "Generic (PLEG): container finished" podID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerID="ececf67d53902a9b966673ca7f27a15081ae6b1c9e919c8faf1db18a797844a9" exitCode=6
Jul 17 11:28:10.193795 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:10.193684    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" event={"ID":"22e9babf-cd4a-4a28-8afb-382f3c13f391","Type":"ContainerDied","Data":"ececf67d53902a9b966673ca7f27a15081ae6b1c9e919c8faf1db18a797844a9"}
Jul 17 11:28:10.193858 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:10.193847    2566 scope.go:117] "RemoveContainer" containerID="ececf67d53902a9b966673ca7f27a15081ae6b1c9e919c8faf1db18a797844a9"
Jul 17 11:28:11.200968 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:11.200871    2566 generic.go:356] "Generic (PLEG): container finished" podID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerID="dd4f105d01ee372d3e53e90736b939e19c767fc581ab8991b040c809f407b22d" exitCode=6
Jul 17 11:28:11.200968 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:11.200953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" event={"ID":"22e9babf-cd4a-4a28-8afb-382f3c13f391","Type":"ContainerDied","Data":"dd4f105d01ee372d3e53e90736b939e19c767fc581ab8991b040c809f407b22d"}
Jul 17 11:28:11.201491 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:11.201011    2566 scope.go:117] "RemoveContainer" containerID="ececf67d53902a9b966673ca7f27a15081ae6b1c9e919c8faf1db18a797844a9"
Jul 17 11:28:11.201491 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:11.201181    2566 scope.go:117] "RemoveContainer" containerID="dd4f105d01ee372d3e53e90736b939e19c767fc581ab8991b040c809f407b22d"
Jul 17 11:28:11.201491 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:11.201447    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx_odh-ai-gateway-infra(22e9babf-cd4a-4a28-8afb-382f3c13f391)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391"
Jul 17 11:28:12.208546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:12.208522    2566 scope.go:117] "RemoveContainer" containerID="dd4f105d01ee372d3e53e90736b939e19c767fc581ab8991b040c809f407b22d"
Jul 17 11:28:12.208920 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:12.208698    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx_odh-ai-gateway-infra(22e9babf-cd4a-4a28-8afb-382f3c13f391)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391"
Jul 17 11:28:13.214956 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:13.214931    2566 generic.go:356] "Generic (PLEG): container finished" podID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerID="f848e79ca95ad41e9aa54e867c80ad000ee50f2b9bba65bcd98fab0b929834cc" exitCode=0
Jul 17 11:28:13.215300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:13.215009    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" event={"ID":"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c","Type":"ContainerDied","Data":"f848e79ca95ad41e9aa54e867c80ad000ee50f2b9bba65bcd98fab0b929834cc"}
Jul 17 11:28:13.215426 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:13.215412    2566 scope.go:117] "RemoveContainer" containerID="f848e79ca95ad41e9aa54e867c80ad000ee50f2b9bba65bcd98fab0b929834cc"
Jul 17 11:28:13.216413 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:13.216383    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:28:13.216487 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:13.216470    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:28:13.216527 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:13.216512    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:28:14.222279 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:14.222245    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" event={"ID":"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c","Type":"ContainerStarted","Data":"d99057f4a0f134b00aa206992e4e78f7183f78e243cbf436d7c311fa5057f328"}
Jul 17 11:28:14.681999 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:14.681963    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:14.683140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:14.683116    2566 patch_prober.go:28] interesting pod/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 17 11:28:14.683205 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:14.683168    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 17 11:28:15.681899 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:15.681866    2566 patch_prober.go:28] interesting pod/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 17 11:28:15.682274 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:15.681916    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 17 11:28:16.681615 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:16.681574    2566 patch_prober.go:28] interesting pod/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 17 11:28:16.681834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:16.681626    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 17 11:28:17.141849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:17.141804    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:17.682074 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:17.682040    2566 patch_prober.go:28] interesting pod/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 17 11:28:17.682248 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:17.682092    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 17 11:28:18.685654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:18.685626    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:18.686549 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:18.686527    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:28:21.651642 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:21.651616    2566 scope.go:117] "RemoveContainer" containerID="32b53313e01e40a8deca2002e2b751a12375b9af46eedceb114da5257ba213a4"
Jul 17 11:28:22.266790 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:22.266764    2566 generic.go:356] "Generic (PLEG): container finished" podID="1e68c051-9514-4695-967a-3d59d19daed8" containerID="c548a98ef1ed7750672c2ee014e95496aa9da958b3d4ec6bd328438b00cb8a24" exitCode=7
Jul 17 11:28:22.266896 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:22.266833    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl" event={"ID":"1e68c051-9514-4695-967a-3d59d19daed8","Type":"ContainerDied","Data":"c548a98ef1ed7750672c2ee014e95496aa9da958b3d4ec6bd328438b00cb8a24"}
Jul 17 11:28:22.266938 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:22.266893    2566 scope.go:117] "RemoveContainer" containerID="32b53313e01e40a8deca2002e2b751a12375b9af46eedceb114da5257ba213a4"
Jul 17 11:28:22.267154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:22.267133    2566 scope.go:117] "RemoveContainer" containerID="c548a98ef1ed7750672c2ee014e95496aa9da958b3d4ec6bd328438b00cb8a24"
Jul 17 11:28:22.267380 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:22.267360    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-znrfl_odh-ai-gateway-infra(1e68c051-9514-4695-967a-3d59d19daed8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl" podUID="1e68c051-9514-4695-967a-3d59d19daed8"
Jul 17 11:28:23.298069 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:23.298038    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"]
Jul 17 11:28:23.429791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:23.429762    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"
Jul 17 11:28:23.530674 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:23.530645    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e68c051-9514-4695-967a-3d59d19daed8-maas-api-service-ca\") pod \"1e68c051-9514-4695-967a-3d59d19daed8\" (UID: \"1e68c051-9514-4695-967a-3d59d19daed8\") "
Jul 17 11:28:23.531057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:23.531037    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e68c051-9514-4695-967a-3d59d19daed8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e68c051-9514-4695-967a-3d59d19daed8" (UID: "1e68c051-9514-4695-967a-3d59d19daed8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:28:23.632276 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:23.632249    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e68c051-9514-4695-967a-3d59d19daed8-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:28:23.652585 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:23.652562    2566 scope.go:117] "RemoveContainer" containerID="dd4f105d01ee372d3e53e90736b939e19c767fc581ab8991b040c809f407b22d"
Jul 17 11:28:24.279651 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:24.279567    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"
Jul 17 11:28:24.279651 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:24.279605    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl" event={"ID":"1e68c051-9514-4695-967a-3d59d19daed8","Type":"ContainerDied","Data":"6534d0034b9f01bc97c2cebb31c67941c7ddbfe860592e4732251f6aca0570c9"}
Jul 17 11:28:24.279910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:24.279666    2566 scope.go:117] "RemoveContainer" containerID="c548a98ef1ed7750672c2ee014e95496aa9da958b3d4ec6bd328438b00cb8a24"
Jul 17 11:28:24.281652 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:24.281622    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" event={"ID":"22e9babf-cd4a-4a28-8afb-382f3c13f391","Type":"ContainerStarted","Data":"f8cb309d1fc38ba6e4f0a7d31ab76b737cf857113402dba6ed4fc14ebc797ef5"}
Jul 17 11:28:24.299660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:24.299634    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"]
Jul 17 11:28:24.307172 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:24.307149    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-znrfl"]
Jul 17 11:28:25.288441 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:25.288412    2566 generic.go:356] "Generic (PLEG): container finished" podID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerID="f8cb309d1fc38ba6e4f0a7d31ab76b737cf857113402dba6ed4fc14ebc797ef5" exitCode=6
Jul 17 11:28:25.288650 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:25.288476    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" event={"ID":"22e9babf-cd4a-4a28-8afb-382f3c13f391","Type":"ContainerDied","Data":"f8cb309d1fc38ba6e4f0a7d31ab76b737cf857113402dba6ed4fc14ebc797ef5"}
Jul 17 11:28:25.288650 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:25.288536    2566 scope.go:117] "RemoveContainer" containerID="dd4f105d01ee372d3e53e90736b939e19c767fc581ab8991b040c809f407b22d"
Jul 17 11:28:25.288856 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:25.288839    2566 scope.go:117] "RemoveContainer" containerID="f8cb309d1fc38ba6e4f0a7d31ab76b737cf857113402dba6ed4fc14ebc797ef5"
Jul 17 11:28:25.289083 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:25.289061    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx_odh-ai-gateway-infra(22e9babf-cd4a-4a28-8afb-382f3c13f391)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391"
Jul 17 11:28:25.333850 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:25.333821    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"]
Jul 17 11:28:25.657433 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:25.657403    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e68c051-9514-4695-967a-3d59d19daed8" path="/var/lib/kubelet/pods/1e68c051-9514-4695-967a-3d59d19daed8/volumes"
Jul 17 11:28:26.429523 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:26.429501    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"
Jul 17 11:28:26.556275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:26.556194    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e9babf-cd4a-4a28-8afb-382f3c13f391-maas-api-service-ca\") pod \"22e9babf-cd4a-4a28-8afb-382f3c13f391\" (UID: \"22e9babf-cd4a-4a28-8afb-382f3c13f391\") "
Jul 17 11:28:26.556589 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:26.556559    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22e9babf-cd4a-4a28-8afb-382f3c13f391-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "22e9babf-cd4a-4a28-8afb-382f3c13f391" (UID: "22e9babf-cd4a-4a28-8afb-382f3c13f391"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:28:26.656689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:26.656661    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e9babf-cd4a-4a28-8afb-382f3c13f391-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:28:27.301559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:27.301535    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"
Jul 17 11:28:27.301767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:27.301566    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx" event={"ID":"22e9babf-cd4a-4a28-8afb-382f3c13f391","Type":"ContainerDied","Data":"653e19dd2a98c5a0170613642d07cd68525290a847395dc4893ac35746621722"}
Jul 17 11:28:27.301767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:27.301625    2566 scope.go:117] "RemoveContainer" containerID="f8cb309d1fc38ba6e4f0a7d31ab76b737cf857113402dba6ed4fc14ebc797ef5"
Jul 17 11:28:27.325580 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:27.325554    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"]
Jul 17 11:28:27.330200 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:27.330178    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-pfgsx"]
Jul 17 11:28:27.657761 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:27.657730    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" path="/var/lib/kubelet/pods/22e9babf-cd4a-4a28-8afb-382f3c13f391/volumes"
Jul 17 11:28:37.791596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.791559    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"]
Jul 17 11:28:37.792070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.791937    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e68c051-9514-4695-967a-3d59d19daed8" containerName="revoke-keys"
Jul 17 11:28:37.792070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.791949    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e68c051-9514-4695-967a-3d59d19daed8" containerName="revoke-keys"
Jul 17 11:28:37.792070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.791956    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e68c051-9514-4695-967a-3d59d19daed8" containerName="revoke-keys"
Jul 17 11:28:37.792070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.791962    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e68c051-9514-4695-967a-3d59d19daed8" containerName="revoke-keys"
Jul 17 11:28:37.792070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.791970    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerName="revoke-keys"
Jul 17 11:28:37.792070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.791976    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerName="revoke-keys"
Jul 17 11:28:37.792070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.791984    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerName="revoke-keys"
Jul 17 11:28:37.792070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.791991    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerName="revoke-keys"
Jul 17 11:28:37.792325 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.792077    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerName="revoke-keys"
Jul 17 11:28:37.792325 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.792087    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e68c051-9514-4695-967a-3d59d19daed8" containerName="revoke-keys"
Jul 17 11:28:37.792325 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.792093    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e68c051-9514-4695-967a-3d59d19daed8" containerName="revoke-keys"
Jul 17 11:28:37.792325 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.792099    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerName="revoke-keys"
Jul 17 11:28:37.795225 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.795208    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"
Jul 17 11:28:37.798021 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.797997    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:28:37.798157 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.798070    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:28:37.805267 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.805242    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"]
Jul 17 11:28:37.857102 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.857070    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f15d914b-b42a-451f-89db-792a79c54a94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq\" (UID: \"f15d914b-b42a-451f-89db-792a79c54a94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"
Jul 17 11:28:37.957933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.957899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f15d914b-b42a-451f-89db-792a79c54a94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq\" (UID: \"f15d914b-b42a-451f-89db-792a79c54a94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"
Jul 17 11:28:37.958566 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:37.958542    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f15d914b-b42a-451f-89db-792a79c54a94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq\" (UID: \"f15d914b-b42a-451f-89db-792a79c54a94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"
Jul 17 11:28:38.106719 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:38.106673    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"
Jul 17 11:28:38.258502 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:38.258477    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"]
Jul 17 11:28:38.261149 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:28:38.261122    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15d914b_b42a_451f_89db_792a79c54a94.slice/crio-8c077521a0ac4a7be727bd1df4ab757321783bdc1197d73623c7b8828124b129 WatchSource:0}: Error finding container 8c077521a0ac4a7be727bd1df4ab757321783bdc1197d73623c7b8828124b129: Status 404 returned error can't find the container with id 8c077521a0ac4a7be727bd1df4ab757321783bdc1197d73623c7b8828124b129
Jul 17 11:28:38.357680 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:38.357612    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq" event={"ID":"f15d914b-b42a-451f-89db-792a79c54a94","Type":"ContainerStarted","Data":"8c077521a0ac4a7be727bd1df4ab757321783bdc1197d73623c7b8828124b129"}
Jul 17 11:28:39.370696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:39.370663    2566 generic.go:356] "Generic (PLEG): container finished" podID="f15d914b-b42a-451f-89db-792a79c54a94" containerID="51e7f1409f26a097c47d6de0a0f5f1aba33b61a210d6d6be77c1ca29225f0c30" exitCode=6
Jul 17 11:28:39.371208 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:39.370759    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq" event={"ID":"f15d914b-b42a-451f-89db-792a79c54a94","Type":"ContainerDied","Data":"51e7f1409f26a097c47d6de0a0f5f1aba33b61a210d6d6be77c1ca29225f0c30"}
Jul 17 11:28:39.371208 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:39.370892    2566 scope.go:117] "RemoveContainer" containerID="51e7f1409f26a097c47d6de0a0f5f1aba33b61a210d6d6be77c1ca29225f0c30"
Jul 17 11:28:40.377912 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:40.377881    2566 generic.go:356] "Generic (PLEG): container finished" podID="f15d914b-b42a-451f-89db-792a79c54a94" containerID="532b2a42ab9d7bea5c4e483d8dfafb35bf05ede2ccb2aaa68f45f27139ae09fe" exitCode=6
Jul 17 11:28:40.378390 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:40.377948    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq" event={"ID":"f15d914b-b42a-451f-89db-792a79c54a94","Type":"ContainerDied","Data":"532b2a42ab9d7bea5c4e483d8dfafb35bf05ede2ccb2aaa68f45f27139ae09fe"}
Jul 17 11:28:40.378390 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:40.377994    2566 scope.go:117] "RemoveContainer" containerID="51e7f1409f26a097c47d6de0a0f5f1aba33b61a210d6d6be77c1ca29225f0c30"
Jul 17 11:28:40.378390 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:40.378214    2566 scope.go:117] "RemoveContainer" containerID="532b2a42ab9d7bea5c4e483d8dfafb35bf05ede2ccb2aaa68f45f27139ae09fe"
Jul 17 11:28:40.378560 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:40.378441    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq_odh-ai-gateway-infra(f15d914b-b42a-451f-89db-792a79c54a94)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq" podUID="f15d914b-b42a-451f-89db-792a79c54a94"
Jul 17 11:28:41.384840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:41.384812    2566 scope.go:117] "RemoveContainer" containerID="532b2a42ab9d7bea5c4e483d8dfafb35bf05ede2ccb2aaa68f45f27139ae09fe"
Jul 17 11:28:41.385238 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:41.385036    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq_odh-ai-gateway-infra(f15d914b-b42a-451f-89db-792a79c54a94)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq" podUID="f15d914b-b42a-451f-89db-792a79c54a94"
Jul 17 11:28:56.652473 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.652432    2566 scope.go:117] "RemoveContainer" containerID="532b2a42ab9d7bea5c4e483d8dfafb35bf05ede2ccb2aaa68f45f27139ae09fe"
Jul 17 11:28:56.731421 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.731390    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"]
Jul 17 11:28:56.731872 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.731852    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerName="revoke-keys"
Jul 17 11:28:56.731925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.731876    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerName="revoke-keys"
Jul 17 11:28:56.731960 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.731926    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e68c051-9514-4695-967a-3d59d19daed8" containerName="revoke-keys"
Jul 17 11:28:56.731960 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.731937    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e68c051-9514-4695-967a-3d59d19daed8" containerName="revoke-keys"
Jul 17 11:28:56.732031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.732019    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e68c051-9514-4695-967a-3d59d19daed8" containerName="revoke-keys"
Jul 17 11:28:56.732067 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.732035    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="22e9babf-cd4a-4a28-8afb-382f3c13f391" containerName="revoke-keys"
Jul 17 11:28:56.736536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.736519    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"
Jul 17 11:28:56.743387 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.743361    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"]
Jul 17 11:28:56.829960 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.829931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2628ad4-a9eb-4433-ad04-9f8febcbc1dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-fk5zj\" (UID: \"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"
Jul 17 11:28:56.931400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.931374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2628ad4-a9eb-4433-ad04-9f8febcbc1dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-fk5zj\" (UID: \"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"
Jul 17 11:28:56.931987 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:56.931969    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2628ad4-a9eb-4433-ad04-9f8febcbc1dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-fk5zj\" (UID: \"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"
Jul 17 11:28:57.048055 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:57.048029    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"
Jul 17 11:28:57.174876 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:57.174853    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"]
Jul 17 11:28:57.177464 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:28:57.177436    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2628ad4_a9eb_4433_ad04_9f8febcbc1dd.slice/crio-094bbaeb9dea849ba35e67a6032053c71e0c041df51f70aed55b93b37c84e99f WatchSource:0}: Error finding container 094bbaeb9dea849ba35e67a6032053c71e0c041df51f70aed55b93b37c84e99f: Status 404 returned error can't find the container with id 094bbaeb9dea849ba35e67a6032053c71e0c041df51f70aed55b93b37c84e99f
Jul 17 11:28:57.462011 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:57.461943    2566 generic.go:356] "Generic (PLEG): container finished" podID="f15d914b-b42a-451f-89db-792a79c54a94" containerID="a903a7edbf3a35ab65d0f0cee501d2310b9eee86c685bf602e893d64d6ae02d3" exitCode=6
Jul 17 11:28:57.462130 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:57.462019    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq" event={"ID":"f15d914b-b42a-451f-89db-792a79c54a94","Type":"ContainerDied","Data":"a903a7edbf3a35ab65d0f0cee501d2310b9eee86c685bf602e893d64d6ae02d3"}
Jul 17 11:28:57.462130 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:57.462083    2566 scope.go:117] "RemoveContainer" containerID="532b2a42ab9d7bea5c4e483d8dfafb35bf05ede2ccb2aaa68f45f27139ae09fe"
Jul 17 11:28:57.462448 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:57.462413    2566 scope.go:117] "RemoveContainer" containerID="a903a7edbf3a35ab65d0f0cee501d2310b9eee86c685bf602e893d64d6ae02d3"
Jul 17 11:28:57.462650 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:57.462626    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq_odh-ai-gateway-infra(f15d914b-b42a-451f-89db-792a79c54a94)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq" podUID="f15d914b-b42a-451f-89db-792a79c54a94"
Jul 17 11:28:57.463539 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:57.463514    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" event={"ID":"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd","Type":"ContainerStarted","Data":"d0cccf2d2e3606cb834f4aa9584bf8e1ed1920be6ee75a06c08d367d13ab644b"}
Jul 17 11:28:57.463620 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:57.463568    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" event={"ID":"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd","Type":"ContainerStarted","Data":"094bbaeb9dea849ba35e67a6032053c71e0c041df51f70aed55b93b37c84e99f"}
Jul 17 11:28:57.501100 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:57.501051    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" podStartSLOduration=1.501036673 podStartE2EDuration="1.501036673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:28:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:28:57.499274074 +0000 UTC m=+1920.356358020" watchObservedRunningTime="2026-07-17 11:28:57.501036673 +0000 UTC m=+1920.358120596"
Jul 17 11:28:58.471507 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:58.471463    2566 generic.go:356] "Generic (PLEG): container finished" podID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerID="d0cccf2d2e3606cb834f4aa9584bf8e1ed1920be6ee75a06c08d367d13ab644b" exitCode=7
Jul 17 11:28:58.471940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:58.471549    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" event={"ID":"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd","Type":"ContainerDied","Data":"d0cccf2d2e3606cb834f4aa9584bf8e1ed1920be6ee75a06c08d367d13ab644b"}
Jul 17 11:28:58.471940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:58.471679    2566 scope.go:117] "RemoveContainer" containerID="d0cccf2d2e3606cb834f4aa9584bf8e1ed1920be6ee75a06c08d367d13ab644b"
Jul 17 11:28:58.496491 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:58.496462    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"]
Jul 17 11:28:58.632045 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:58.632023    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"
Jul 17 11:28:58.745209 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:58.745138    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f15d914b-b42a-451f-89db-792a79c54a94-maas-api-service-ca\") pod \"f15d914b-b42a-451f-89db-792a79c54a94\" (UID: \"f15d914b-b42a-451f-89db-792a79c54a94\") "
Jul 17 11:28:58.745462 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:58.745439    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f15d914b-b42a-451f-89db-792a79c54a94-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f15d914b-b42a-451f-89db-792a79c54a94" (UID: "f15d914b-b42a-451f-89db-792a79c54a94"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:28:58.846647 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:58.846621    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f15d914b-b42a-451f-89db-792a79c54a94-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:28:59.478692 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.478663    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"
Jul 17 11:28:59.478692 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.478675    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq" event={"ID":"f15d914b-b42a-451f-89db-792a79c54a94","Type":"ContainerDied","Data":"8c077521a0ac4a7be727bd1df4ab757321783bdc1197d73623c7b8828124b129"}
Jul 17 11:28:59.479237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.478750    2566 scope.go:117] "RemoveContainer" containerID="a903a7edbf3a35ab65d0f0cee501d2310b9eee86c685bf602e893d64d6ae02d3"
Jul 17 11:28:59.480447 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.480424    2566 generic.go:356] "Generic (PLEG): container finished" podID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerID="144c64ef3fc12202b323d65283cfdbf865193fbad35a2ced84d9b25d64d47931" exitCode=7
Jul 17 11:28:59.480563 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.480485    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" event={"ID":"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd","Type":"ContainerDied","Data":"144c64ef3fc12202b323d65283cfdbf865193fbad35a2ced84d9b25d64d47931"}
Jul 17 11:28:59.480778 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.480761    2566 scope.go:117] "RemoveContainer" containerID="144c64ef3fc12202b323d65283cfdbf865193fbad35a2ced84d9b25d64d47931"
Jul 17 11:28:59.481040 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:28:59.481013    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-fk5zj_odh-ai-gateway-infra(e2628ad4-a9eb-4433-ad04-9f8febcbc1dd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd"
Jul 17 11:28:59.487673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.487659    2566 scope.go:117] "RemoveContainer" containerID="d0cccf2d2e3606cb834f4aa9584bf8e1ed1920be6ee75a06c08d367d13ab644b"
Jul 17 11:28:59.541389 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.541364    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"]
Jul 17 11:28:59.547807 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.547784    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-lwzjq"]
Jul 17 11:28:59.658239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:28:59.658217    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f15d914b-b42a-451f-89db-792a79c54a94" path="/var/lib/kubelet/pods/f15d914b-b42a-451f-89db-792a79c54a94/volumes"
Jul 17 11:29:00.487903 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:00.487877    2566 scope.go:117] "RemoveContainer" containerID="144c64ef3fc12202b323d65283cfdbf865193fbad35a2ced84d9b25d64d47931"
Jul 17 11:29:00.488282 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:29:00.488065    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-fk5zj_odh-ai-gateway-infra(e2628ad4-a9eb-4433-ad04-9f8febcbc1dd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd"
Jul 17 11:29:11.652023 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:11.651990    2566 scope.go:117] "RemoveContainer" containerID="144c64ef3fc12202b323d65283cfdbf865193fbad35a2ced84d9b25d64d47931"
Jul 17 11:29:12.547224 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:12.547137    2566 generic.go:356] "Generic (PLEG): container finished" podID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerID="4c190744f282ec11e8beebf3fbd8f67372b91649a93f082fcbc77e026759dc5a" exitCode=7
Jul 17 11:29:12.547224 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:12.547177    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" event={"ID":"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd","Type":"ContainerDied","Data":"4c190744f282ec11e8beebf3fbd8f67372b91649a93f082fcbc77e026759dc5a"}
Jul 17 11:29:12.547224 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:12.547226    2566 scope.go:117] "RemoveContainer" containerID="144c64ef3fc12202b323d65283cfdbf865193fbad35a2ced84d9b25d64d47931"
Jul 17 11:29:12.547517 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:12.547494    2566 scope.go:117] "RemoveContainer" containerID="4c190744f282ec11e8beebf3fbd8f67372b91649a93f082fcbc77e026759dc5a"
Jul 17 11:29:12.547730 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:29:12.547688    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-fk5zj_odh-ai-gateway-infra(e2628ad4-a9eb-4433-ad04-9f8febcbc1dd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd"
Jul 17 11:29:13.574688 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:13.574657    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"]
Jul 17 11:29:13.713864 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:13.713839    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"
Jul 17 11:29:13.872253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:13.872169    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2628ad4-a9eb-4433-ad04-9f8febcbc1dd-maas-api-service-ca\") pod \"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd\" (UID: \"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd\") "
Jul 17 11:29:13.872511 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:13.872491    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2628ad4-a9eb-4433-ad04-9f8febcbc1dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" (UID: "e2628ad4-a9eb-4433-ad04-9f8febcbc1dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:29:13.972919 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:13.972889    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2628ad4-a9eb-4433-ad04-9f8febcbc1dd-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:29:14.559684 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:14.559654    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj" event={"ID":"e2628ad4-a9eb-4433-ad04-9f8febcbc1dd","Type":"ContainerDied","Data":"094bbaeb9dea849ba35e67a6032053c71e0c041df51f70aed55b93b37c84e99f"}
Jul 17 11:29:14.559880 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:14.559689    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"
Jul 17 11:29:14.559880 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:14.559733    2566 scope.go:117] "RemoveContainer" containerID="4c190744f282ec11e8beebf3fbd8f67372b91649a93f082fcbc77e026759dc5a"
Jul 17 11:29:14.586983 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:14.586961    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"]
Jul 17 11:29:14.593864 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:14.593841    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-fk5zj"]
Jul 17 11:29:15.656523 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:15.656493    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" path="/var/lib/kubelet/pods/e2628ad4-a9eb-4433-ad04-9f8febcbc1dd/volumes"
Jul 17 11:29:46.776237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776207    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"]
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776551    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776562    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776579    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15d914b-b42a-451f-89db-792a79c54a94" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776584    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15d914b-b42a-451f-89db-792a79c54a94" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776595    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15d914b-b42a-451f-89db-792a79c54a94" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776601    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15d914b-b42a-451f-89db-792a79c54a94" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776606    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776611    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776621    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15d914b-b42a-451f-89db-792a79c54a94" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776625    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15d914b-b42a-451f-89db-792a79c54a94" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776633    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776638    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776700    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f15d914b-b42a-451f-89db-792a79c54a94" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776721    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776727    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerName="revoke-keys"
Jul 17 11:29:46.776734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.776735    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f15d914b-b42a-451f-89db-792a79c54a94" containerName="revoke-keys"
Jul 17 11:29:46.779694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.779675    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"
Jul 17 11:29:46.782339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.782318    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:29:46.782443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.782375    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:29:46.787233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.787180    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"]
Jul 17 11:29:46.858277 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.858248    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e109e7a0-0749-407f-be2a-2c794ffa7369-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-xnsqf\" (UID: \"e109e7a0-0749-407f-be2a-2c794ffa7369\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"
Jul 17 11:29:46.959547 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.959510    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e109e7a0-0749-407f-be2a-2c794ffa7369-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-xnsqf\" (UID: \"e109e7a0-0749-407f-be2a-2c794ffa7369\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"
Jul 17 11:29:46.960118 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:46.960100    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e109e7a0-0749-407f-be2a-2c794ffa7369-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-xnsqf\" (UID: \"e109e7a0-0749-407f-be2a-2c794ffa7369\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"
Jul 17 11:29:47.091051 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:47.091026    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"
Jul 17 11:29:47.219798 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:47.219773    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"]
Jul 17 11:29:47.221718 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:29:47.221686    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode109e7a0_0749_407f_be2a_2c794ffa7369.slice/crio-ce00ddfd8e552f4f86487745674fc457c83b9d5960c2dbf4c711a33f5e166556 WatchSource:0}: Error finding container ce00ddfd8e552f4f86487745674fc457c83b9d5960c2dbf4c711a33f5e166556: Status 404 returned error can't find the container with id ce00ddfd8e552f4f86487745674fc457c83b9d5960c2dbf4c711a33f5e166556
Jul 17 11:29:47.715436 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:47.715335    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" event={"ID":"e109e7a0-0749-407f-be2a-2c794ffa7369","Type":"ContainerStarted","Data":"34cfc5430e065dded537cbc31697bf8b31d85541f7d295fc5dbe76c2090e9e0f"}
Jul 17 11:29:47.715436 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:47.715401    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" event={"ID":"e109e7a0-0749-407f-be2a-2c794ffa7369","Type":"ContainerStarted","Data":"ce00ddfd8e552f4f86487745674fc457c83b9d5960c2dbf4c711a33f5e166556"}
Jul 17 11:29:47.733908 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:47.733247    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" podStartSLOduration=1.733228338 podStartE2EDuration="1.733228338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:29:47.731213484 +0000 UTC m=+1970.588297421" watchObservedRunningTime="2026-07-17 11:29:47.733228338 +0000 UTC m=+1970.590312284"
Jul 17 11:29:48.720986 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:48.720953    2566 generic.go:356] "Generic (PLEG): container finished" podID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerID="34cfc5430e065dded537cbc31697bf8b31d85541f7d295fc5dbe76c2090e9e0f" exitCode=7
Jul 17 11:29:48.721419 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:48.721029    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" event={"ID":"e109e7a0-0749-407f-be2a-2c794ffa7369","Type":"ContainerDied","Data":"34cfc5430e065dded537cbc31697bf8b31d85541f7d295fc5dbe76c2090e9e0f"}
Jul 17 11:29:48.721419 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:48.721185    2566 scope.go:117] "RemoveContainer" containerID="34cfc5430e065dded537cbc31697bf8b31d85541f7d295fc5dbe76c2090e9e0f"
Jul 17 11:29:49.728354 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:49.728272    2566 generic.go:356] "Generic (PLEG): container finished" podID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerID="df3ae2a5ce2830d985e4fc98a95abbba8bcfc2005a58caddd3fd033ce04c688a" exitCode=7
Jul 17 11:29:49.728354 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:49.728323    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" event={"ID":"e109e7a0-0749-407f-be2a-2c794ffa7369","Type":"ContainerDied","Data":"df3ae2a5ce2830d985e4fc98a95abbba8bcfc2005a58caddd3fd033ce04c688a"}
Jul 17 11:29:49.728837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:49.728369    2566 scope.go:117] "RemoveContainer" containerID="34cfc5430e065dded537cbc31697bf8b31d85541f7d295fc5dbe76c2090e9e0f"
Jul 17 11:29:49.728837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:49.728549    2566 scope.go:117] "RemoveContainer" containerID="df3ae2a5ce2830d985e4fc98a95abbba8bcfc2005a58caddd3fd033ce04c688a"
Jul 17 11:29:49.728837 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:29:49.728775    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-xnsqf_odh-ai-gateway-infra(e109e7a0-0749-407f-be2a-2c794ffa7369)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369"
Jul 17 11:29:50.735950 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:50.735866    2566 scope.go:117] "RemoveContainer" containerID="df3ae2a5ce2830d985e4fc98a95abbba8bcfc2005a58caddd3fd033ce04c688a"
Jul 17 11:29:50.736411 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:29:50.736121    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-xnsqf_odh-ai-gateway-infra(e109e7a0-0749-407f-be2a-2c794ffa7369)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369"
Jul 17 11:29:57.926946 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:57.926912    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"]
Jul 17 11:29:57.927498 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:57.927477    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2628ad4-a9eb-4433-ad04-9f8febcbc1dd" containerName="revoke-keys"
Jul 17 11:29:57.927591 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:57.927502    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f15d914b-b42a-451f-89db-792a79c54a94" containerName="revoke-keys"
Jul 17 11:29:57.931782 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:57.931762    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"
Jul 17 11:29:57.947542 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:57.947519    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"]
Jul 17 11:29:58.058510 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:58.058478    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffc017f3-3623-4980-8867-b1ccca1c9d0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq\" (UID: \"ffc017f3-3623-4980-8867-b1ccca1c9d0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"
Jul 17 11:29:58.159117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:58.159033    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffc017f3-3623-4980-8867-b1ccca1c9d0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq\" (UID: \"ffc017f3-3623-4980-8867-b1ccca1c9d0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"
Jul 17 11:29:58.159742 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:58.159700    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffc017f3-3623-4980-8867-b1ccca1c9d0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq\" (UID: \"ffc017f3-3623-4980-8867-b1ccca1c9d0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"
Jul 17 11:29:58.241956 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:58.241934    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"
Jul 17 11:29:58.377899 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:58.377872    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"]
Jul 17 11:29:58.378846 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:29:58.378820    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc017f3_3623_4980_8867_b1ccca1c9d0e.slice/crio-aba123a585ae928cfb5a46a409ab8560e33abe4712ac63f5424fdd0cb30bfb70 WatchSource:0}: Error finding container aba123a585ae928cfb5a46a409ab8560e33abe4712ac63f5424fdd0cb30bfb70: Status 404 returned error can't find the container with id aba123a585ae928cfb5a46a409ab8560e33abe4712ac63f5424fdd0cb30bfb70
Jul 17 11:29:58.776760 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:58.776669    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" event={"ID":"ffc017f3-3623-4980-8867-b1ccca1c9d0e","Type":"ContainerStarted","Data":"fcbd046afef22c0f55414358cea33aa9d85537abdfad7aa7233ad0c367bd09d0"}
Jul 17 11:29:58.776760 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:58.776742    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" event={"ID":"ffc017f3-3623-4980-8867-b1ccca1c9d0e","Type":"ContainerStarted","Data":"aba123a585ae928cfb5a46a409ab8560e33abe4712ac63f5424fdd0cb30bfb70"}
Jul 17 11:29:59.785339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:59.785302    2566 generic.go:356] "Generic (PLEG): container finished" podID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerID="fcbd046afef22c0f55414358cea33aa9d85537abdfad7aa7233ad0c367bd09d0" exitCode=6
Jul 17 11:29:59.785728 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:59.785370    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" event={"ID":"ffc017f3-3623-4980-8867-b1ccca1c9d0e","Type":"ContainerDied","Data":"fcbd046afef22c0f55414358cea33aa9d85537abdfad7aa7233ad0c367bd09d0"}
Jul 17 11:29:59.785728 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:29:59.785532    2566 scope.go:117] "RemoveContainer" containerID="fcbd046afef22c0f55414358cea33aa9d85537abdfad7aa7233ad0c367bd09d0"
Jul 17 11:30:00.150877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.150851    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k"]
Jul 17 11:30:00.154249 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.154233    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k"
Jul 17 11:30:00.164079 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.164058    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k"
Jul 17 11:30:00.168446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.168424    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k"]
Jul 17 11:30:00.339362 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.339340    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k"]
Jul 17 11:30:00.341698 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:30:00.341673    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7a9d80_bdcf_4158_a2aa_a913c9168672.slice/crio-5db28f44c4a088ffd2b8f595d783ab6df68a85ca719ac830dbb761b28cd2ad1d WatchSource:0}: Error finding container 5db28f44c4a088ffd2b8f595d783ab6df68a85ca719ac830dbb761b28cd2ad1d: Status 404 returned error can't find the container with id 5db28f44c4a088ffd2b8f595d783ab6df68a85ca719ac830dbb761b28cd2ad1d
Jul 17 11:30:00.791450 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.791417    2566 generic.go:356] "Generic (PLEG): container finished" podID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerID="8bb1df89ca265684f745f380f5cc7d2cd5af0efe04915305333ebe908f68fc11" exitCode=6
Jul 17 11:30:00.791902 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.791500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" event={"ID":"ffc017f3-3623-4980-8867-b1ccca1c9d0e","Type":"ContainerDied","Data":"8bb1df89ca265684f745f380f5cc7d2cd5af0efe04915305333ebe908f68fc11"}
Jul 17 11:30:00.791902 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.791555    2566 scope.go:117] "RemoveContainer" containerID="fcbd046afef22c0f55414358cea33aa9d85537abdfad7aa7233ad0c367bd09d0"
Jul 17 11:30:00.791902 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.791668    2566 scope.go:117] "RemoveContainer" containerID="8bb1df89ca265684f745f380f5cc7d2cd5af0efe04915305333ebe908f68fc11"
Jul 17 11:30:00.791902 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:30:00.791877    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq_odh-ai-gateway-infra(ffc017f3-3623-4980-8867-b1ccca1c9d0e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e"
Jul 17 11:30:00.793119 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.793091    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k" event={"ID":"ea7a9d80-bdcf-4158-a2aa-a913c9168672","Type":"ContainerStarted","Data":"8c435fd36160c9d3b87fbbe5a8931ce6024e08462c96aa04a2611b5f3403c667"}
Jul 17 11:30:00.793217 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.793139    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k" event={"ID":"ea7a9d80-bdcf-4158-a2aa-a913c9168672","Type":"ContainerStarted","Data":"5db28f44c4a088ffd2b8f595d783ab6df68a85ca719ac830dbb761b28cd2ad1d"}
Jul 17 11:30:00.830842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:00.830801    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k" podStartSLOduration=0.830789057 podStartE2EDuration="830.789057ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:30:00.82958355 +0000 UTC m=+1983.686667495" watchObservedRunningTime="2026-07-17 11:30:00.830789057 +0000 UTC m=+1983.687873001"
Jul 17 11:30:01.799396 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:01.799368    2566 scope.go:117] "RemoveContainer" containerID="8bb1df89ca265684f745f380f5cc7d2cd5af0efe04915305333ebe908f68fc11"
Jul 17 11:30:01.799884 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:30:01.799577    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq_odh-ai-gateway-infra(ffc017f3-3623-4980-8867-b1ccca1c9d0e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e"
Jul 17 11:30:01.800681 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:01.800658    2566 generic.go:356] "Generic (PLEG): container finished" podID="ea7a9d80-bdcf-4158-a2aa-a913c9168672" containerID="8c435fd36160c9d3b87fbbe5a8931ce6024e08462c96aa04a2611b5f3403c667" exitCode=0
Jul 17 11:30:01.800804 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:01.800737    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k" event={"ID":"ea7a9d80-bdcf-4158-a2aa-a913c9168672","Type":"ContainerDied","Data":"8c435fd36160c9d3b87fbbe5a8931ce6024e08462c96aa04a2611b5f3403c667"}
Jul 17 11:30:02.938342 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:02.938314    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k"
Jul 17 11:30:03.813499 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:03.813471    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k"
Jul 17 11:30:03.813657 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:03.813468    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738130-v6v8k" event={"ID":"ea7a9d80-bdcf-4158-a2aa-a913c9168672","Type":"ContainerDied","Data":"5db28f44c4a088ffd2b8f595d783ab6df68a85ca719ac830dbb761b28cd2ad1d"}
Jul 17 11:30:03.813657 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:03.813598    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5db28f44c4a088ffd2b8f595d783ab6df68a85ca719ac830dbb761b28cd2ad1d"
Jul 17 11:30:04.652343 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:04.652311    2566 scope.go:117] "RemoveContainer" containerID="df3ae2a5ce2830d985e4fc98a95abbba8bcfc2005a58caddd3fd033ce04c688a"
Jul 17 11:30:05.824771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:05.824741    2566 generic.go:356] "Generic (PLEG): container finished" podID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerID="19f8553311b03179a8671006ea53c115e0bac0f0226e3a058e4cedfdc664a39a" exitCode=7
Jul 17 11:30:05.825184 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:05.824815    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" event={"ID":"e109e7a0-0749-407f-be2a-2c794ffa7369","Type":"ContainerDied","Data":"19f8553311b03179a8671006ea53c115e0bac0f0226e3a058e4cedfdc664a39a"}
Jul 17 11:30:05.825184 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:05.824872    2566 scope.go:117] "RemoveContainer" containerID="df3ae2a5ce2830d985e4fc98a95abbba8bcfc2005a58caddd3fd033ce04c688a"
Jul 17 11:30:05.825317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:05.825199    2566 scope.go:117] "RemoveContainer" containerID="19f8553311b03179a8671006ea53c115e0bac0f0226e3a058e4cedfdc664a39a"
Jul 17 11:30:05.825463 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:30:05.825441    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-xnsqf_odh-ai-gateway-infra(e109e7a0-0749-407f-be2a-2c794ffa7369)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369"
Jul 17 11:30:06.856717 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:06.856684    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"]
Jul 17 11:30:06.990718 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:06.990689    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"
Jul 17 11:30:07.137443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:07.137375    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e109e7a0-0749-407f-be2a-2c794ffa7369-maas-api-service-ca\") pod \"e109e7a0-0749-407f-be2a-2c794ffa7369\" (UID: \"e109e7a0-0749-407f-be2a-2c794ffa7369\") "
Jul 17 11:30:07.137722 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:07.137686    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e109e7a0-0749-407f-be2a-2c794ffa7369-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e109e7a0-0749-407f-be2a-2c794ffa7369" (UID: "e109e7a0-0749-407f-be2a-2c794ffa7369"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:30:07.238654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:07.238627    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e109e7a0-0749-407f-be2a-2c794ffa7369-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:30:07.840374 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:07.840343    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"
Jul 17 11:30:07.840538 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:07.840343    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf" event={"ID":"e109e7a0-0749-407f-be2a-2c794ffa7369","Type":"ContainerDied","Data":"ce00ddfd8e552f4f86487745674fc457c83b9d5960c2dbf4c711a33f5e166556"}
Jul 17 11:30:07.840538 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:07.840481    2566 scope.go:117] "RemoveContainer" containerID="19f8553311b03179a8671006ea53c115e0bac0f0226e3a058e4cedfdc664a39a"
Jul 17 11:30:07.863914 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:07.863890    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"]
Jul 17 11:30:07.875342 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:07.875321    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-xnsqf"]
Jul 17 11:30:09.657513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:09.657472    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" path="/var/lib/kubelet/pods/e109e7a0-0749-407f-be2a-2c794ffa7369/volumes"
Jul 17 11:30:15.652197 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:15.652167    2566 scope.go:117] "RemoveContainer" containerID="8bb1df89ca265684f745f380f5cc7d2cd5af0efe04915305333ebe908f68fc11"
Jul 17 11:30:15.882595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:15.882565    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" event={"ID":"ffc017f3-3623-4980-8867-b1ccca1c9d0e","Type":"ContainerStarted","Data":"2d6f698931ce54518325bf6fe2b6b786d617e077b83260082adfde4d48598462"}
Jul 17 11:30:15.907856 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:15.907779    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" podStartSLOduration=18.907764242 podStartE2EDuration="18.907764242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:29:58.802391543 +0000 UTC m=+1981.659475488" watchObservedRunningTime="2026-07-17 11:30:15.907764242 +0000 UTC m=+1998.764848186"
Jul 17 11:30:16.889601 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:16.889558    2566 generic.go:356] "Generic (PLEG): container finished" podID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerID="2d6f698931ce54518325bf6fe2b6b786d617e077b83260082adfde4d48598462" exitCode=6
Jul 17 11:30:16.890029 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:16.889633    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" event={"ID":"ffc017f3-3623-4980-8867-b1ccca1c9d0e","Type":"ContainerDied","Data":"2d6f698931ce54518325bf6fe2b6b786d617e077b83260082adfde4d48598462"}
Jul 17 11:30:16.890029 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:16.889692    2566 scope.go:117] "RemoveContainer" containerID="8bb1df89ca265684f745f380f5cc7d2cd5af0efe04915305333ebe908f68fc11"
Jul 17 11:30:16.890029 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:16.889961    2566 scope.go:117] "RemoveContainer" containerID="2d6f698931ce54518325bf6fe2b6b786d617e077b83260082adfde4d48598462"
Jul 17 11:30:16.890180 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:30:16.890158    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq_odh-ai-gateway-infra(ffc017f3-3623-4980-8867-b1ccca1c9d0e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e"
Jul 17 11:30:16.920507 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:16.920484    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"]
Jul 17 11:30:18.030256 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:18.030232    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"
Jul 17 11:30:18.132520 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:18.132492    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffc017f3-3623-4980-8867-b1ccca1c9d0e-maas-api-service-ca\") pod \"ffc017f3-3623-4980-8867-b1ccca1c9d0e\" (UID: \"ffc017f3-3623-4980-8867-b1ccca1c9d0e\") "
Jul 17 11:30:18.132848 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:18.132828    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffc017f3-3623-4980-8867-b1ccca1c9d0e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ffc017f3-3623-4980-8867-b1ccca1c9d0e" (UID: "ffc017f3-3623-4980-8867-b1ccca1c9d0e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:30:18.233408 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:18.233345    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffc017f3-3623-4980-8867-b1ccca1c9d0e-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:30:18.902104 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:18.902074    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"
Jul 17 11:30:18.902104 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:18.902084    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq" event={"ID":"ffc017f3-3623-4980-8867-b1ccca1c9d0e","Type":"ContainerDied","Data":"aba123a585ae928cfb5a46a409ab8560e33abe4712ac63f5424fdd0cb30bfb70"}
Jul 17 11:30:18.902333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:18.902140    2566 scope.go:117] "RemoveContainer" containerID="2d6f698931ce54518325bf6fe2b6b786d617e077b83260082adfde4d48598462"
Jul 17 11:30:18.926724 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:18.926681    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"]
Jul 17 11:30:18.933750 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:18.933727    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-nf2mq"]
Jul 17 11:30:19.661885 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:19.659145    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" path="/var/lib/kubelet/pods/ffc017f3-3623-4980-8867-b1ccca1c9d0e/volumes"
Jul 17 11:30:46.845414 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.845382    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"]
Jul 17 11:30:46.845925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.845906    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerName="revoke-keys"
Jul 17 11:30:46.846005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.845928    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerName="revoke-keys"
Jul 17 11:30:46.846005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.845947    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea7a9d80-bdcf-4158-a2aa-a913c9168672" containerName="cleanup"
Jul 17 11:30:46.846005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.845955    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea7a9d80-bdcf-4158-a2aa-a913c9168672" containerName="cleanup"
Jul 17 11:30:46.846005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.845973    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerName="revoke-keys"
Jul 17 11:30:46.846005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.845982    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846009    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846018    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846028    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846036    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846133    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea7a9d80-bdcf-4158-a2aa-a913c9168672" containerName="cleanup"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846147    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846158    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846167    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846177    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846188    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerName="revoke-keys"
Jul 17 11:30:46.846253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.846197    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerName="revoke-keys"
Jul 17 11:30:46.849409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.849386    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"
Jul 17 11:30:46.852436 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.852416    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:30:46.852524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.852422    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:30:46.863721 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.863689    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"]
Jul 17 11:30:46.980371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:46.980336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42bc6a47-d355-4aa6-b410-4d635e852da8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-v8ck9\" (UID: \"42bc6a47-d355-4aa6-b410-4d635e852da8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"
Jul 17 11:30:47.081068 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:47.081032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42bc6a47-d355-4aa6-b410-4d635e852da8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-v8ck9\" (UID: \"42bc6a47-d355-4aa6-b410-4d635e852da8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"
Jul 17 11:30:47.081675 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:47.081653    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42bc6a47-d355-4aa6-b410-4d635e852da8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-v8ck9\" (UID: \"42bc6a47-d355-4aa6-b410-4d635e852da8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"
Jul 17 11:30:47.158993 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:47.158917    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"
Jul 17 11:30:47.505543 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:47.505521    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"]
Jul 17 11:30:47.508048 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:30:47.508019    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42bc6a47_d355_4aa6_b410_4d635e852da8.slice/crio-b4ab773b0d2d54b605077bbbca2558e9b814fd6d9abb8ed5fa66c72b1485bf2b WatchSource:0}: Error finding container b4ab773b0d2d54b605077bbbca2558e9b814fd6d9abb8ed5fa66c72b1485bf2b: Status 404 returned error can't find the container with id b4ab773b0d2d54b605077bbbca2558e9b814fd6d9abb8ed5fa66c72b1485bf2b
Jul 17 11:30:48.045257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:48.045173    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" event={"ID":"42bc6a47-d355-4aa6-b410-4d635e852da8","Type":"ContainerStarted","Data":"3a6ea644ec2ecfb8052b656f2f42495aa4c3dafc40b19b157da1abf70b940df7"}
Jul 17 11:30:48.045257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:48.045223    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" event={"ID":"42bc6a47-d355-4aa6-b410-4d635e852da8","Type":"ContainerStarted","Data":"b4ab773b0d2d54b605077bbbca2558e9b814fd6d9abb8ed5fa66c72b1485bf2b"}
Jul 17 11:30:48.072577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:48.072532    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" podStartSLOduration=2.072517862 podStartE2EDuration="2.072517862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:30:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:30:48.072385937 +0000 UTC m=+2030.929469882" watchObservedRunningTime="2026-07-17 11:30:48.072517862 +0000 UTC m=+2030.929601807"
Jul 17 11:30:49.051054 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:49.051014    2566 generic.go:356] "Generic (PLEG): container finished" podID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerID="3a6ea644ec2ecfb8052b656f2f42495aa4c3dafc40b19b157da1abf70b940df7" exitCode=7
Jul 17 11:30:49.051530 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:49.051098    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" event={"ID":"42bc6a47-d355-4aa6-b410-4d635e852da8","Type":"ContainerDied","Data":"3a6ea644ec2ecfb8052b656f2f42495aa4c3dafc40b19b157da1abf70b940df7"}
Jul 17 11:30:49.051530 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:49.051291    2566 scope.go:117] "RemoveContainer" containerID="3a6ea644ec2ecfb8052b656f2f42495aa4c3dafc40b19b157da1abf70b940df7"
Jul 17 11:30:50.058361 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:50.058265    2566 generic.go:356] "Generic (PLEG): container finished" podID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerID="ca5191e784ed87d88d63d72648b0e9a79fa76b20a24bdcc81df34763ed67e03a" exitCode=7
Jul 17 11:30:50.058361 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:50.058348    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" event={"ID":"42bc6a47-d355-4aa6-b410-4d635e852da8","Type":"ContainerDied","Data":"ca5191e784ed87d88d63d72648b0e9a79fa76b20a24bdcc81df34763ed67e03a"}
Jul 17 11:30:50.058853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:50.058408    2566 scope.go:117] "RemoveContainer" containerID="3a6ea644ec2ecfb8052b656f2f42495aa4c3dafc40b19b157da1abf70b940df7"
Jul 17 11:30:50.058853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:50.058572    2566 scope.go:117] "RemoveContainer" containerID="ca5191e784ed87d88d63d72648b0e9a79fa76b20a24bdcc81df34763ed67e03a"
Jul 17 11:30:50.058941 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:30:50.058850    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-v8ck9_odh-ai-gateway-infra(42bc6a47-d355-4aa6-b410-4d635e852da8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8"
Jul 17 11:30:51.065339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:30:51.065309    2566 scope.go:117] "RemoveContainer" containerID="ca5191e784ed87d88d63d72648b0e9a79fa76b20a24bdcc81df34763ed67e03a"
Jul 17 11:30:51.065752 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:30:51.065511    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-v8ck9_odh-ai-gateway-infra(42bc6a47-d355-4aa6-b410-4d635e852da8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8"
Jul 17 11:31:05.652304 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:05.652272    2566 scope.go:117] "RemoveContainer" containerID="ca5191e784ed87d88d63d72648b0e9a79fa76b20a24bdcc81df34763ed67e03a"
Jul 17 11:31:06.142754 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:06.142697    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" event={"ID":"42bc6a47-d355-4aa6-b410-4d635e852da8","Type":"ContainerStarted","Data":"1f34ecd9de58f49aa7edf29bf7371f486343290de6483bb088d1b62e928f8191"}
Jul 17 11:31:07.149228 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:07.149193    2566 generic.go:356] "Generic (PLEG): container finished" podID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerID="1f34ecd9de58f49aa7edf29bf7371f486343290de6483bb088d1b62e928f8191" exitCode=7
Jul 17 11:31:07.149655 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:07.149266    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" event={"ID":"42bc6a47-d355-4aa6-b410-4d635e852da8","Type":"ContainerDied","Data":"1f34ecd9de58f49aa7edf29bf7371f486343290de6483bb088d1b62e928f8191"}
Jul 17 11:31:07.149655 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:07.149337    2566 scope.go:117] "RemoveContainer" containerID="ca5191e784ed87d88d63d72648b0e9a79fa76b20a24bdcc81df34763ed67e03a"
Jul 17 11:31:07.149655 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:07.149596    2566 scope.go:117] "RemoveContainer" containerID="1f34ecd9de58f49aa7edf29bf7371f486343290de6483bb088d1b62e928f8191"
Jul 17 11:31:07.149831 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:31:07.149809    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-v8ck9_odh-ai-gateway-infra(42bc6a47-d355-4aa6-b410-4d635e852da8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8"
Jul 17 11:31:07.177313 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:07.177290    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"]
Jul 17 11:31:08.295329 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:08.295303    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"
Jul 17 11:31:08.375431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:08.375410    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42bc6a47-d355-4aa6-b410-4d635e852da8-maas-api-service-ca\") pod \"42bc6a47-d355-4aa6-b410-4d635e852da8\" (UID: \"42bc6a47-d355-4aa6-b410-4d635e852da8\") "
Jul 17 11:31:08.375757 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:08.375738    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42bc6a47-d355-4aa6-b410-4d635e852da8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42bc6a47-d355-4aa6-b410-4d635e852da8" (UID: "42bc6a47-d355-4aa6-b410-4d635e852da8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:31:08.476748 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:08.476670    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42bc6a47-d355-4aa6-b410-4d635e852da8-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:31:09.160873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:09.160845    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"
Jul 17 11:31:09.160873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:09.160868    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9" event={"ID":"42bc6a47-d355-4aa6-b410-4d635e852da8","Type":"ContainerDied","Data":"b4ab773b0d2d54b605077bbbca2558e9b814fd6d9abb8ed5fa66c72b1485bf2b"}
Jul 17 11:31:09.161146 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:09.160922    2566 scope.go:117] "RemoveContainer" containerID="1f34ecd9de58f49aa7edf29bf7371f486343290de6483bb088d1b62e928f8191"
Jul 17 11:31:09.183768 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:09.183743    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"]
Jul 17 11:31:09.192110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:09.192087    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-v8ck9"]
Jul 17 11:31:09.656785 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:09.656755    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" path="/var/lib/kubelet/pods/42bc6a47-d355-4aa6-b410-4d635e852da8/volumes"
Jul 17 11:31:46.919658 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.919623    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"]
Jul 17 11:31:46.920101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.919985    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerName="revoke-keys"
Jul 17 11:31:46.920101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.919996    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerName="revoke-keys"
Jul 17 11:31:46.920101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.920013    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerName="revoke-keys"
Jul 17 11:31:46.920101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.920019    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc017f3-3623-4980-8867-b1ccca1c9d0e" containerName="revoke-keys"
Jul 17 11:31:46.920101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.920027    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerName="revoke-keys"
Jul 17 11:31:46.920101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.920033    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerName="revoke-keys"
Jul 17 11:31:46.920101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.920043    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerName="revoke-keys"
Jul 17 11:31:46.920101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.920048    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e109e7a0-0749-407f-be2a-2c794ffa7369" containerName="revoke-keys"
Jul 17 11:31:46.920101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.920104    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerName="revoke-keys"
Jul 17 11:31:46.920377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.920113    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerName="revoke-keys"
Jul 17 11:31:46.920377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.920121    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerName="revoke-keys"
Jul 17 11:31:46.923141 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.923125    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"
Jul 17 11:31:46.926406 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.926390    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:31:46.926499 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.926414    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:31:46.933600 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.933576    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"]
Jul 17 11:31:46.998073 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:46.998044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce478f62-adc6-4f53-afb9-49e223447dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-n5xbb\" (UID: \"ce478f62-adc6-4f53-afb9-49e223447dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"
Jul 17 11:31:47.099396 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:47.099369    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce478f62-adc6-4f53-afb9-49e223447dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-n5xbb\" (UID: \"ce478f62-adc6-4f53-afb9-49e223447dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"
Jul 17 11:31:47.099998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:47.099976    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce478f62-adc6-4f53-afb9-49e223447dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-n5xbb\" (UID: \"ce478f62-adc6-4f53-afb9-49e223447dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"
Jul 17 11:31:47.233739 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:47.233648    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"
Jul 17 11:31:47.366359 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:47.366330    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"]
Jul 17 11:31:47.368330 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:31:47.368302    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce478f62_adc6_4f53_afb9_49e223447dd7.slice/crio-13130483f7a82ad5a148ea0272cae0618a603839357e578129e532e72eb17aac WatchSource:0}: Error finding container 13130483f7a82ad5a148ea0272cae0618a603839357e578129e532e72eb17aac: Status 404 returned error can't find the container with id 13130483f7a82ad5a148ea0272cae0618a603839357e578129e532e72eb17aac
Jul 17 11:31:48.350320 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:48.350286    2566 generic.go:356] "Generic (PLEG): container finished" podID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerID="f41279404dafcda8ec1d6963afeb887c02d0d75d5e0ee4aed92ae36bcf40856e" exitCode=7
Jul 17 11:31:48.350736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:48.350347    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb" event={"ID":"ce478f62-adc6-4f53-afb9-49e223447dd7","Type":"ContainerDied","Data":"f41279404dafcda8ec1d6963afeb887c02d0d75d5e0ee4aed92ae36bcf40856e"}
Jul 17 11:31:48.350736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:48.350409    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb" event={"ID":"ce478f62-adc6-4f53-afb9-49e223447dd7","Type":"ContainerStarted","Data":"13130483f7a82ad5a148ea0272cae0618a603839357e578129e532e72eb17aac"}
Jul 17 11:31:48.350736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:48.350512    2566 scope.go:117] "RemoveContainer" containerID="f41279404dafcda8ec1d6963afeb887c02d0d75d5e0ee4aed92ae36bcf40856e"
Jul 17 11:31:49.357369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:49.357339    2566 generic.go:356] "Generic (PLEG): container finished" podID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerID="d32f4b76ce39f891b55c1b9ba37cf690c503ac467458b5fedd6927604ae0eeaf" exitCode=7
Jul 17 11:31:49.357838 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:49.357428    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb" event={"ID":"ce478f62-adc6-4f53-afb9-49e223447dd7","Type":"ContainerDied","Data":"d32f4b76ce39f891b55c1b9ba37cf690c503ac467458b5fedd6927604ae0eeaf"}
Jul 17 11:31:49.357838 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:49.357489    2566 scope.go:117] "RemoveContainer" containerID="f41279404dafcda8ec1d6963afeb887c02d0d75d5e0ee4aed92ae36bcf40856e"
Jul 17 11:31:49.357838 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:49.357742    2566 scope.go:117] "RemoveContainer" containerID="d32f4b76ce39f891b55c1b9ba37cf690c503ac467458b5fedd6927604ae0eeaf"
Jul 17 11:31:49.358013 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:31:49.357934    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-n5xbb_odh-ai-gateway-infra(ce478f62-adc6-4f53-afb9-49e223447dd7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7"
Jul 17 11:31:50.363742 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:50.363690    2566 scope.go:117] "RemoveContainer" containerID="d32f4b76ce39f891b55c1b9ba37cf690c503ac467458b5fedd6927604ae0eeaf"
Jul 17 11:31:50.364137 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:31:50.363933    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-n5xbb_odh-ai-gateway-infra(ce478f62-adc6-4f53-afb9-49e223447dd7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7"
Jul 17 11:31:57.788050 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:57.788023    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:31:57.791191 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:57.791167    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:31:57.793907 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:57.793886    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:31:57.796918 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:31:57.796904    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:32:01.651689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:01.651664    2566 scope.go:117] "RemoveContainer" containerID="d32f4b76ce39f891b55c1b9ba37cf690c503ac467458b5fedd6927604ae0eeaf"
Jul 17 11:32:02.421358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:02.421326    2566 generic.go:356] "Generic (PLEG): container finished" podID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerID="df88430f7b90653eed5104c421090fa53e46b79f2f06d8882301afc81530eb6c" exitCode=7
Jul 17 11:32:02.421583 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:02.421403    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb" event={"ID":"ce478f62-adc6-4f53-afb9-49e223447dd7","Type":"ContainerDied","Data":"df88430f7b90653eed5104c421090fa53e46b79f2f06d8882301afc81530eb6c"}
Jul 17 11:32:02.421583 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:02.421465    2566 scope.go:117] "RemoveContainer" containerID="d32f4b76ce39f891b55c1b9ba37cf690c503ac467458b5fedd6927604ae0eeaf"
Jul 17 11:32:02.421783 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:02.421762    2566 scope.go:117] "RemoveContainer" containerID="df88430f7b90653eed5104c421090fa53e46b79f2f06d8882301afc81530eb6c"
Jul 17 11:32:02.422029 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:32:02.422006    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-n5xbb_odh-ai-gateway-infra(ce478f62-adc6-4f53-afb9-49e223447dd7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7"
Jul 17 11:32:03.457524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:03.457487    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"]
Jul 17 11:32:03.594025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:03.594000    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"
Jul 17 11:32:03.742735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:03.742638    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce478f62-adc6-4f53-afb9-49e223447dd7-maas-api-service-ca\") pod \"ce478f62-adc6-4f53-afb9-49e223447dd7\" (UID: \"ce478f62-adc6-4f53-afb9-49e223447dd7\") "
Jul 17 11:32:03.743019 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:03.742997    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce478f62-adc6-4f53-afb9-49e223447dd7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ce478f62-adc6-4f53-afb9-49e223447dd7" (UID: "ce478f62-adc6-4f53-afb9-49e223447dd7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:32:03.843868 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:03.843831    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce478f62-adc6-4f53-afb9-49e223447dd7-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:32:04.433915 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:04.433878    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb" event={"ID":"ce478f62-adc6-4f53-afb9-49e223447dd7","Type":"ContainerDied","Data":"13130483f7a82ad5a148ea0272cae0618a603839357e578129e532e72eb17aac"}
Jul 17 11:32:04.433915 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:04.433907    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"
Jul 17 11:32:04.434171 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:04.433962    2566 scope.go:117] "RemoveContainer" containerID="df88430f7b90653eed5104c421090fa53e46b79f2f06d8882301afc81530eb6c"
Jul 17 11:32:04.457682 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:04.457655    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"]
Jul 17 11:32:04.461890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:04.461868    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-n5xbb"]
Jul 17 11:32:05.657584 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:05.657554    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" path="/var/lib/kubelet/pods/ce478f62-adc6-4f53-afb9-49e223447dd7/volumes"
Jul 17 11:32:36.943662 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.943628    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"]
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.943983    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerName="revoke-keys"
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.943995    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerName="revoke-keys"
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.944002    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerName="revoke-keys"
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.944008    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerName="revoke-keys"
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.944024    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerName="revoke-keys"
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.944030    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerName="revoke-keys"
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.944043    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerName="revoke-keys"
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.944048    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="42bc6a47-d355-4aa6-b410-4d635e852da8" containerName="revoke-keys"
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.944118    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerName="revoke-keys"
Jul 17 11:32:36.944134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.944127    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerName="revoke-keys"
Jul 17 11:32:36.947012 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.946995    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"
Jul 17 11:32:36.950504 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.950478    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:32:36.950638 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.950618    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:32:36.957681 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:36.957662    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"]
Jul 17 11:32:37.036832 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:37.036804    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/777f8f27-bd15-4f0f-bc4f-5eec425489b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-vhvtf\" (UID: \"777f8f27-bd15-4f0f-bc4f-5eec425489b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"
Jul 17 11:32:37.138196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:37.138170    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/777f8f27-bd15-4f0f-bc4f-5eec425489b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-vhvtf\" (UID: \"777f8f27-bd15-4f0f-bc4f-5eec425489b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"
Jul 17 11:32:37.138790 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:37.138771    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/777f8f27-bd15-4f0f-bc4f-5eec425489b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-vhvtf\" (UID: \"777f8f27-bd15-4f0f-bc4f-5eec425489b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"
Jul 17 11:32:37.257549 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:37.257475    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"
Jul 17 11:32:37.390002 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:37.389977    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"]
Jul 17 11:32:37.391833 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:32:37.391804    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777f8f27_bd15_4f0f_bc4f_5eec425489b7.slice/crio-39445e93dadba3219ee22001774e15bb6624dac0e864a7f0ec7020668fec976b WatchSource:0}: Error finding container 39445e93dadba3219ee22001774e15bb6624dac0e864a7f0ec7020668fec976b: Status 404 returned error can't find the container with id 39445e93dadba3219ee22001774e15bb6624dac0e864a7f0ec7020668fec976b
Jul 17 11:32:37.593310 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:37.593280    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf" event={"ID":"777f8f27-bd15-4f0f-bc4f-5eec425489b7","Type":"ContainerStarted","Data":"39445e93dadba3219ee22001774e15bb6624dac0e864a7f0ec7020668fec976b"}
Jul 17 11:32:38.599599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:38.599564    2566 generic.go:356] "Generic (PLEG): container finished" podID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerID="1f84d86849d7a40ad56226eca8d2ffebe45664288380e9769914b06df3daca49" exitCode=7
Jul 17 11:32:38.599599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:38.599599    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf" event={"ID":"777f8f27-bd15-4f0f-bc4f-5eec425489b7","Type":"ContainerDied","Data":"1f84d86849d7a40ad56226eca8d2ffebe45664288380e9769914b06df3daca49"}
Jul 17 11:32:38.600154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:38.599870    2566 scope.go:117] "RemoveContainer" containerID="1f84d86849d7a40ad56226eca8d2ffebe45664288380e9769914b06df3daca49"
Jul 17 11:32:39.606136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:39.606042    2566 generic.go:356] "Generic (PLEG): container finished" podID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerID="8c8b0f5c4275d4b436b44194a5f40ee327177b5084536a62dd9ef9b792552cea" exitCode=7
Jul 17 11:32:39.606574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:39.606132    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf" event={"ID":"777f8f27-bd15-4f0f-bc4f-5eec425489b7","Type":"ContainerDied","Data":"8c8b0f5c4275d4b436b44194a5f40ee327177b5084536a62dd9ef9b792552cea"}
Jul 17 11:32:39.606574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:39.606241    2566 scope.go:117] "RemoveContainer" containerID="1f84d86849d7a40ad56226eca8d2ffebe45664288380e9769914b06df3daca49"
Jul 17 11:32:39.606574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:39.606441    2566 scope.go:117] "RemoveContainer" containerID="8c8b0f5c4275d4b436b44194a5f40ee327177b5084536a62dd9ef9b792552cea"
Jul 17 11:32:39.606767 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:32:39.606736    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-vhvtf_odh-ai-gateway-infra(777f8f27-bd15-4f0f-bc4f-5eec425489b7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7"
Jul 17 11:32:40.613144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:40.613115    2566 scope.go:117] "RemoveContainer" containerID="8c8b0f5c4275d4b436b44194a5f40ee327177b5084536a62dd9ef9b792552cea"
Jul 17 11:32:40.613554 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:32:40.613340    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-vhvtf_odh-ai-gateway-infra(777f8f27-bd15-4f0f-bc4f-5eec425489b7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7"
Jul 17 11:32:51.652540 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:51.652462    2566 scope.go:117] "RemoveContainer" containerID="8c8b0f5c4275d4b436b44194a5f40ee327177b5084536a62dd9ef9b792552cea"
Jul 17 11:32:52.674456 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:52.674366    2566 generic.go:356] "Generic (PLEG): container finished" podID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerID="9d6eb92f976134c94ebc735bc71a5b20b93595b5f37618bc30d22d869c29693e" exitCode=7
Jul 17 11:32:52.674929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:52.674445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf" event={"ID":"777f8f27-bd15-4f0f-bc4f-5eec425489b7","Type":"ContainerDied","Data":"9d6eb92f976134c94ebc735bc71a5b20b93595b5f37618bc30d22d869c29693e"}
Jul 17 11:32:52.674929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:52.674517    2566 scope.go:117] "RemoveContainer" containerID="8c8b0f5c4275d4b436b44194a5f40ee327177b5084536a62dd9ef9b792552cea"
Jul 17 11:32:52.674929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:52.674783    2566 scope.go:117] "RemoveContainer" containerID="9d6eb92f976134c94ebc735bc71a5b20b93595b5f37618bc30d22d869c29693e"
Jul 17 11:32:52.675053 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:32:52.675010    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-vhvtf_odh-ai-gateway-infra(777f8f27-bd15-4f0f-bc4f-5eec425489b7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7"
Jul 17 11:32:53.712635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:53.712603    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"]
Jul 17 11:32:53.863115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:53.863088    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"
Jul 17 11:32:53.890429 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:53.890400    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/777f8f27-bd15-4f0f-bc4f-5eec425489b7-maas-api-service-ca\") pod \"777f8f27-bd15-4f0f-bc4f-5eec425489b7\" (UID: \"777f8f27-bd15-4f0f-bc4f-5eec425489b7\") "
Jul 17 11:32:53.890738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:53.890699    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/777f8f27-bd15-4f0f-bc4f-5eec425489b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "777f8f27-bd15-4f0f-bc4f-5eec425489b7" (UID: "777f8f27-bd15-4f0f-bc4f-5eec425489b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:32:53.991698 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:53.991597    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/777f8f27-bd15-4f0f-bc4f-5eec425489b7-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:32:54.685839 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:54.685803    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf" event={"ID":"777f8f27-bd15-4f0f-bc4f-5eec425489b7","Type":"ContainerDied","Data":"39445e93dadba3219ee22001774e15bb6624dac0e864a7f0ec7020668fec976b"}
Jul 17 11:32:54.686032 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:54.685860    2566 scope.go:117] "RemoveContainer" containerID="9d6eb92f976134c94ebc735bc71a5b20b93595b5f37618bc30d22d869c29693e"
Jul 17 11:32:54.686032 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:54.685820    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"
Jul 17 11:32:54.714346 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:54.714322    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"]
Jul 17 11:32:54.723133 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:54.723110    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vhvtf"]
Jul 17 11:32:55.658521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:32:55.658484    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" path="/var/lib/kubelet/pods/777f8f27-bd15-4f0f-bc4f-5eec425489b7/volumes"
Jul 17 11:33:18.088602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.088569    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"]
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.088945    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerName="revoke-keys"
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.088957    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerName="revoke-keys"
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.088972    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerName="revoke-keys"
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.088977    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerName="revoke-keys"
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.088986    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerName="revoke-keys"
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.088991    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerName="revoke-keys"
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.089046    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce478f62-adc6-4f53-afb9-49e223447dd7" containerName="revoke-keys"
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.089054    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerName="revoke-keys"
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.089060    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerName="revoke-keys"
Jul 17 11:33:18.090969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.089067    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="777f8f27-bd15-4f0f-bc4f-5eec425489b7" containerName="revoke-keys"
Jul 17 11:33:18.091916 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.091900    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"
Jul 17 11:33:18.094471 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.094450    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:33:18.094567 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.094553    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:33:18.099962 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.099895    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"]
Jul 17 11:33:18.204775 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.204741    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8181cd31-b9ac-4130-83ab-0595af33e096-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd\" (UID: \"8181cd31-b9ac-4130-83ab-0595af33e096\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"
Jul 17 11:33:18.305371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.305329    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8181cd31-b9ac-4130-83ab-0595af33e096-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd\" (UID: \"8181cd31-b9ac-4130-83ab-0595af33e096\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"
Jul 17 11:33:18.306058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.306036    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8181cd31-b9ac-4130-83ab-0595af33e096-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd\" (UID: \"8181cd31-b9ac-4130-83ab-0595af33e096\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"
Jul 17 11:33:18.403931 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.403830    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"
Jul 17 11:33:18.539413 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.539390    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"]
Jul 17 11:33:18.541339 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:33:18.541299    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8181cd31_b9ac_4130_83ab_0595af33e096.slice/crio-5b5d675c4f5cd597e0db90fa9c9b04400d8e017e769c7279524005cf38e2c9ef WatchSource:0}: Error finding container 5b5d675c4f5cd597e0db90fa9c9b04400d8e017e769c7279524005cf38e2c9ef: Status 404 returned error can't find the container with id 5b5d675c4f5cd597e0db90fa9c9b04400d8e017e769c7279524005cf38e2c9ef
Jul 17 11:33:18.803181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:18.803149    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" event={"ID":"8181cd31-b9ac-4130-83ab-0595af33e096","Type":"ContainerStarted","Data":"5b5d675c4f5cd597e0db90fa9c9b04400d8e017e769c7279524005cf38e2c9ef"}
Jul 17 11:33:19.815749 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:19.815685    2566 generic.go:356] "Generic (PLEG): container finished" podID="8181cd31-b9ac-4130-83ab-0595af33e096" containerID="443767b64db9f4386fb172b09d76c984279f8294b7413842f252978dee7dbbd9" exitCode=6
Jul 17 11:33:19.816215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:19.815831    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" event={"ID":"8181cd31-b9ac-4130-83ab-0595af33e096","Type":"ContainerDied","Data":"443767b64db9f4386fb172b09d76c984279f8294b7413842f252978dee7dbbd9"}
Jul 17 11:33:19.816215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:19.816075    2566 scope.go:117] "RemoveContainer" containerID="443767b64db9f4386fb172b09d76c984279f8294b7413842f252978dee7dbbd9"
Jul 17 11:33:20.822609 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:20.822577    2566 generic.go:356] "Generic (PLEG): container finished" podID="8181cd31-b9ac-4130-83ab-0595af33e096" containerID="7bb6386b4d441d659ce23dd0247dfcf25cb7962a1171464d86c73bd1e70af21c" exitCode=6
Jul 17 11:33:20.823020 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:20.822662    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" event={"ID":"8181cd31-b9ac-4130-83ab-0595af33e096","Type":"ContainerDied","Data":"7bb6386b4d441d659ce23dd0247dfcf25cb7962a1171464d86c73bd1e70af21c"}
Jul 17 11:33:20.823020 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:20.822746    2566 scope.go:117] "RemoveContainer" containerID="443767b64db9f4386fb172b09d76c984279f8294b7413842f252978dee7dbbd9"
Jul 17 11:33:20.823020 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:20.822888    2566 scope.go:117] "RemoveContainer" containerID="7bb6386b4d441d659ce23dd0247dfcf25cb7962a1171464d86c73bd1e70af21c"
Jul 17 11:33:20.823170 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:33:20.823134    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd_odh-ai-gateway-infra(8181cd31-b9ac-4130-83ab-0595af33e096)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" podUID="8181cd31-b9ac-4130-83ab-0595af33e096"
Jul 17 11:33:21.829646 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:21.829614    2566 scope.go:117] "RemoveContainer" containerID="7bb6386b4d441d659ce23dd0247dfcf25cb7962a1171464d86c73bd1e70af21c"
Jul 17 11:33:21.830084 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:33:21.829846    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd_odh-ai-gateway-infra(8181cd31-b9ac-4130-83ab-0595af33e096)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" podUID="8181cd31-b9ac-4130-83ab-0595af33e096"
Jul 17 11:33:33.651921 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:33.651888    2566 scope.go:117] "RemoveContainer" containerID="7bb6386b4d441d659ce23dd0247dfcf25cb7962a1171464d86c73bd1e70af21c"
Jul 17 11:33:33.894646 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:33.894618    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" event={"ID":"8181cd31-b9ac-4130-83ab-0595af33e096","Type":"ContainerStarted","Data":"363d4e209c4af3eadc9336630dba6e372dd5c6dc147c4b912810202459cddf86"}
Jul 17 11:33:33.913076 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:33.912987    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" podStartSLOduration=15.912972519 podStartE2EDuration="15.912972519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:33:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:33:33.91147759 +0000 UTC m=+2196.768561537" watchObservedRunningTime="2026-07-17 11:33:33.912972519 +0000 UTC m=+2196.770056464"
Jul 17 11:33:34.901472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:34.901431    2566 generic.go:356] "Generic (PLEG): container finished" podID="8181cd31-b9ac-4130-83ab-0595af33e096" containerID="363d4e209c4af3eadc9336630dba6e372dd5c6dc147c4b912810202459cddf86" exitCode=6
Jul 17 11:33:34.901969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:34.901503    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" event={"ID":"8181cd31-b9ac-4130-83ab-0595af33e096","Type":"ContainerDied","Data":"363d4e209c4af3eadc9336630dba6e372dd5c6dc147c4b912810202459cddf86"}
Jul 17 11:33:34.901969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:34.901563    2566 scope.go:117] "RemoveContainer" containerID="7bb6386b4d441d659ce23dd0247dfcf25cb7962a1171464d86c73bd1e70af21c"
Jul 17 11:33:34.901969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:34.901847    2566 scope.go:117] "RemoveContainer" containerID="363d4e209c4af3eadc9336630dba6e372dd5c6dc147c4b912810202459cddf86"
Jul 17 11:33:34.902122 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:33:34.902036    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd_odh-ai-gateway-infra(8181cd31-b9ac-4130-83ab-0595af33e096)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" podUID="8181cd31-b9ac-4130-83ab-0595af33e096"
Jul 17 11:33:34.924356 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:34.924330    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"]
Jul 17 11:33:36.039540 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:36.039515    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"
Jul 17 11:33:36.060109 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:36.060084    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8181cd31-b9ac-4130-83ab-0595af33e096-maas-api-service-ca\") pod \"8181cd31-b9ac-4130-83ab-0595af33e096\" (UID: \"8181cd31-b9ac-4130-83ab-0595af33e096\") "
Jul 17 11:33:36.060390 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:36.060371    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8181cd31-b9ac-4130-83ab-0595af33e096-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8181cd31-b9ac-4130-83ab-0595af33e096" (UID: "8181cd31-b9ac-4130-83ab-0595af33e096"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:33:36.160725 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:36.160628    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8181cd31-b9ac-4130-83ab-0595af33e096-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:33:36.914173 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:36.914138    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"
Jul 17 11:33:36.914376 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:36.914139    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd" event={"ID":"8181cd31-b9ac-4130-83ab-0595af33e096","Type":"ContainerDied","Data":"5b5d675c4f5cd597e0db90fa9c9b04400d8e017e769c7279524005cf38e2c9ef"}
Jul 17 11:33:36.914376 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:36.914282    2566 scope.go:117] "RemoveContainer" containerID="363d4e209c4af3eadc9336630dba6e372dd5c6dc147c4b912810202459cddf86"
Jul 17 11:33:36.939557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:36.939529    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"]
Jul 17 11:33:36.944230 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:36.944204    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rvcbd"]
Jul 17 11:33:37.657829 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:37.657795    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" path="/var/lib/kubelet/pods/8181cd31-b9ac-4130-83ab-0595af33e096/volumes"
Jul 17 11:33:38.171020 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.170980    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"]
Jul 17 11:33:38.171432 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.171413    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" containerName="revoke-keys"
Jul 17 11:33:38.171521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.171433    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" containerName="revoke-keys"
Jul 17 11:33:38.171521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.171452    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" containerName="revoke-keys"
Jul 17 11:33:38.171521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.171460    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" containerName="revoke-keys"
Jul 17 11:33:38.171521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.171473    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" containerName="revoke-keys"
Jul 17 11:33:38.171521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.171481    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" containerName="revoke-keys"
Jul 17 11:33:38.171812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.171568    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" containerName="revoke-keys"
Jul 17 11:33:38.171812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.171580    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" containerName="revoke-keys"
Jul 17 11:33:38.171812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.171593    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8181cd31-b9ac-4130-83ab-0595af33e096" containerName="revoke-keys"
Jul 17 11:33:38.175987 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.175963    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"
Jul 17 11:33:38.179005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.178986    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:33:38.179111 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.179009    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:33:38.185363 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.185338    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"]
Jul 17 11:33:38.278120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.278086    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18a031b-7580-4446-86fc-206879c312f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6\" (UID: \"b18a031b-7580-4446-86fc-206879c312f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"
Jul 17 11:33:38.378820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.378784    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18a031b-7580-4446-86fc-206879c312f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6\" (UID: \"b18a031b-7580-4446-86fc-206879c312f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"
Jul 17 11:33:38.379393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.379366    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18a031b-7580-4446-86fc-206879c312f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6\" (UID: \"b18a031b-7580-4446-86fc-206879c312f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"
Jul 17 11:33:38.487289 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.487206    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"
Jul 17 11:33:38.615343 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.615318    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"]
Jul 17 11:33:38.616916 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:33:38.616882    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18a031b_7580_4446_86fc_206879c312f8.slice/crio-885e284a3f05dd0fc237a20f05796a54a91c9f6137c0ed17fc891e3588c0547a WatchSource:0}: Error finding container 885e284a3f05dd0fc237a20f05796a54a91c9f6137c0ed17fc891e3588c0547a: Status 404 returned error can't find the container with id 885e284a3f05dd0fc237a20f05796a54a91c9f6137c0ed17fc891e3588c0547a
Jul 17 11:33:38.927044 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.927006    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" event={"ID":"b18a031b-7580-4446-86fc-206879c312f8","Type":"ContainerStarted","Data":"0aeedfe512abfea2e34ed1b23e75a371493272927713f01a690bc3153ecbd39c"}
Jul 17 11:33:38.927503 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.927058    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" event={"ID":"b18a031b-7580-4446-86fc-206879c312f8","Type":"ContainerStarted","Data":"885e284a3f05dd0fc237a20f05796a54a91c9f6137c0ed17fc891e3588c0547a"}
Jul 17 11:33:38.945138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:38.945087    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" podStartSLOduration=0.945041601 podStartE2EDuration="945.041601ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:33:38.94442935 +0000 UTC m=+2201.801513297" watchObservedRunningTime="2026-07-17 11:33:38.945041601 +0000 UTC m=+2201.802125545"
Jul 17 11:33:39.933063 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:39.933031    2566 generic.go:356] "Generic (PLEG): container finished" podID="b18a031b-7580-4446-86fc-206879c312f8" containerID="0aeedfe512abfea2e34ed1b23e75a371493272927713f01a690bc3153ecbd39c" exitCode=6
Jul 17 11:33:39.933478 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:39.933122    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" event={"ID":"b18a031b-7580-4446-86fc-206879c312f8","Type":"ContainerDied","Data":"0aeedfe512abfea2e34ed1b23e75a371493272927713f01a690bc3153ecbd39c"}
Jul 17 11:33:39.933478 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:39.933292    2566 scope.go:117] "RemoveContainer" containerID="0aeedfe512abfea2e34ed1b23e75a371493272927713f01a690bc3153ecbd39c"
Jul 17 11:33:40.939968 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:40.939934    2566 generic.go:356] "Generic (PLEG): container finished" podID="b18a031b-7580-4446-86fc-206879c312f8" containerID="eeadf5458093b5aa77eccc14a6cd7870774a1b8cd0d0c434032a061a958d01df" exitCode=6
Jul 17 11:33:40.940431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:40.939995    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" event={"ID":"b18a031b-7580-4446-86fc-206879c312f8","Type":"ContainerDied","Data":"eeadf5458093b5aa77eccc14a6cd7870774a1b8cd0d0c434032a061a958d01df"}
Jul 17 11:33:40.940431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:40.940064    2566 scope.go:117] "RemoveContainer" containerID="0aeedfe512abfea2e34ed1b23e75a371493272927713f01a690bc3153ecbd39c"
Jul 17 11:33:40.940431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:40.940317    2566 scope.go:117] "RemoveContainer" containerID="eeadf5458093b5aa77eccc14a6cd7870774a1b8cd0d0c434032a061a958d01df"
Jul 17 11:33:40.940594 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:33:40.940558    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6_odh-ai-gateway-infra(b18a031b-7580-4446-86fc-206879c312f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" podUID="b18a031b-7580-4446-86fc-206879c312f8"
Jul 17 11:33:41.946629 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:41.946602    2566 scope.go:117] "RemoveContainer" containerID="eeadf5458093b5aa77eccc14a6cd7870774a1b8cd0d0c434032a061a958d01df"
Jul 17 11:33:41.947006 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:33:41.946817    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6_odh-ai-gateway-infra(b18a031b-7580-4446-86fc-206879c312f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" podUID="b18a031b-7580-4446-86fc-206879c312f8"
Jul 17 11:33:54.652163 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:54.652129    2566 scope.go:117] "RemoveContainer" containerID="eeadf5458093b5aa77eccc14a6cd7870774a1b8cd0d0c434032a061a958d01df"
Jul 17 11:33:55.011397 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:55.011318    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" event={"ID":"b18a031b-7580-4446-86fc-206879c312f8","Type":"ContainerStarted","Data":"a26c78fc4287618a967c64abc37c8844a7df263dd065a08bd29f74ab72b16e51"}
Jul 17 11:33:56.017918 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:56.017882    2566 generic.go:356] "Generic (PLEG): container finished" podID="b18a031b-7580-4446-86fc-206879c312f8" containerID="a26c78fc4287618a967c64abc37c8844a7df263dd065a08bd29f74ab72b16e51" exitCode=6
Jul 17 11:33:56.018330 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:56.017953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" event={"ID":"b18a031b-7580-4446-86fc-206879c312f8","Type":"ContainerDied","Data":"a26c78fc4287618a967c64abc37c8844a7df263dd065a08bd29f74ab72b16e51"}
Jul 17 11:33:56.018330 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:56.018014    2566 scope.go:117] "RemoveContainer" containerID="eeadf5458093b5aa77eccc14a6cd7870774a1b8cd0d0c434032a061a958d01df"
Jul 17 11:33:56.018330 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:56.018250    2566 scope.go:117] "RemoveContainer" containerID="a26c78fc4287618a967c64abc37c8844a7df263dd065a08bd29f74ab72b16e51"
Jul 17 11:33:56.018506 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:33:56.018486    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6_odh-ai-gateway-infra(b18a031b-7580-4446-86fc-206879c312f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" podUID="b18a031b-7580-4446-86fc-206879c312f8"
Jul 17 11:33:56.042557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:56.042535    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"]
Jul 17 11:33:57.170618 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:57.170590    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"
Jul 17 11:33:57.252670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:57.252635    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18a031b-7580-4446-86fc-206879c312f8-maas-api-service-ca\") pod \"b18a031b-7580-4446-86fc-206879c312f8\" (UID: \"b18a031b-7580-4446-86fc-206879c312f8\") "
Jul 17 11:33:57.253027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:57.253004    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b18a031b-7580-4446-86fc-206879c312f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b18a031b-7580-4446-86fc-206879c312f8" (UID: "b18a031b-7580-4446-86fc-206879c312f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:33:57.353947 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:57.353916    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18a031b-7580-4446-86fc-206879c312f8-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:33:58.030214 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:58.030142    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"
Jul 17 11:33:58.030360 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:58.030144    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6" event={"ID":"b18a031b-7580-4446-86fc-206879c312f8","Type":"ContainerDied","Data":"885e284a3f05dd0fc237a20f05796a54a91c9f6137c0ed17fc891e3588c0547a"}
Jul 17 11:33:58.030360 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:58.030307    2566 scope.go:117] "RemoveContainer" containerID="a26c78fc4287618a967c64abc37c8844a7df263dd065a08bd29f74ab72b16e51"
Jul 17 11:33:58.049220 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:58.049193    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"]
Jul 17 11:33:58.054922 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:58.054891    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-sj8j6"]
Jul 17 11:33:59.657712 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:33:59.657667    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b18a031b-7580-4446-86fc-206879c312f8" path="/var/lib/kubelet/pods/b18a031b-7580-4446-86fc-206879c312f8/volumes"
Jul 17 11:34:06.952185 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:06.952150    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"]
Jul 17 11:34:06.952746 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:06.952338    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy" containerID="cri-o://d99057f4a0f134b00aa206992e4e78f7183f78e243cbf436d7c311fa5057f328" gracePeriod=30
Jul 17 11:34:08.146394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.146358    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"]
Jul 17 11:34:08.146805 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.146740    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18a031b-7580-4446-86fc-206879c312f8" containerName="revoke-keys"
Jul 17 11:34:08.146805 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.146751    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18a031b-7580-4446-86fc-206879c312f8" containerName="revoke-keys"
Jul 17 11:34:08.146805 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.146766    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18a031b-7580-4446-86fc-206879c312f8" containerName="revoke-keys"
Jul 17 11:34:08.146805 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.146772    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18a031b-7580-4446-86fc-206879c312f8" containerName="revoke-keys"
Jul 17 11:34:08.146959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.146831    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b18a031b-7580-4446-86fc-206879c312f8" containerName="revoke-keys"
Jul 17 11:34:08.146959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.146844    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b18a031b-7580-4446-86fc-206879c312f8" containerName="revoke-keys"
Jul 17 11:34:08.149888 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.149871    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"
Jul 17 11:34:08.152895 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.152874    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:34:08.153028 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.152879    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:34:08.159814 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.159789    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"]
Jul 17 11:34:08.248756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.248694    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb83089-7468-4859-8c49-e09ef61261dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv\" (UID: \"2cb83089-7468-4859-8c49-e09ef61261dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"
Jul 17 11:34:08.350106 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.350078    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb83089-7468-4859-8c49-e09ef61261dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv\" (UID: \"2cb83089-7468-4859-8c49-e09ef61261dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"
Jul 17 11:34:08.350672 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.350652    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb83089-7468-4859-8c49-e09ef61261dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv\" (UID: \"2cb83089-7468-4859-8c49-e09ef61261dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"
Jul 17 11:34:08.461373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.461292    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"
Jul 17 11:34:08.592241 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:08.592217    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"]
Jul 17 11:34:08.594482 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:34:08.594453    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb83089_7468_4859_8c49_e09ef61261dc.slice/crio-43f68a47728bb9b09226cd53955bdca8e3bacaaaa1c3a26f142779bcfe2e89b4 WatchSource:0}: Error finding container 43f68a47728bb9b09226cd53955bdca8e3bacaaaa1c3a26f142779bcfe2e89b4: Status 404 returned error can't find the container with id 43f68a47728bb9b09226cd53955bdca8e3bacaaaa1c3a26f142779bcfe2e89b4
Jul 17 11:34:09.085368 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:09.085335    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" event={"ID":"2cb83089-7468-4859-8c49-e09ef61261dc","Type":"ContainerStarted","Data":"a650eb26318518b7ffca25dc40ed4a57aaeec3c57dbb9a80ac6d888f235dc679"}
Jul 17 11:34:09.085569 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:09.085391    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" event={"ID":"2cb83089-7468-4859-8c49-e09ef61261dc","Type":"ContainerStarted","Data":"43f68a47728bb9b09226cd53955bdca8e3bacaaaa1c3a26f142779bcfe2e89b4"}
Jul 17 11:34:09.107766 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:09.107667    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" podStartSLOduration=1.107654287 podStartE2EDuration="1.107654287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:34:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:34:09.105725668 +0000 UTC m=+2231.962809614" watchObservedRunningTime="2026-07-17 11:34:09.107654287 +0000 UTC m=+2231.964738231"
Jul 17 11:34:09.546880 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:09.546800    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:34:10.091631 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:10.091599    2566 generic.go:356] "Generic (PLEG): container finished" podID="2cb83089-7468-4859-8c49-e09ef61261dc" containerID="a650eb26318518b7ffca25dc40ed4a57aaeec3c57dbb9a80ac6d888f235dc679" exitCode=6
Jul 17 11:34:10.091834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:10.091691    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" event={"ID":"2cb83089-7468-4859-8c49-e09ef61261dc","Type":"ContainerDied","Data":"a650eb26318518b7ffca25dc40ed4a57aaeec3c57dbb9a80ac6d888f235dc679"}
Jul 17 11:34:10.091908 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:10.091893    2566 scope.go:117] "RemoveContainer" containerID="a650eb26318518b7ffca25dc40ed4a57aaeec3c57dbb9a80ac6d888f235dc679"
Jul 17 11:34:11.098470 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:11.098436    2566 generic.go:356] "Generic (PLEG): container finished" podID="2cb83089-7468-4859-8c49-e09ef61261dc" containerID="36e195f02a3628b7e987dbafa0c529a101a0a401b9de1cdfd14495ac0ee52847" exitCode=6
Jul 17 11:34:11.098929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:11.098470    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" event={"ID":"2cb83089-7468-4859-8c49-e09ef61261dc","Type":"ContainerDied","Data":"36e195f02a3628b7e987dbafa0c529a101a0a401b9de1cdfd14495ac0ee52847"}
Jul 17 11:34:11.098929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:11.098533    2566 scope.go:117] "RemoveContainer" containerID="a650eb26318518b7ffca25dc40ed4a57aaeec3c57dbb9a80ac6d888f235dc679"
Jul 17 11:34:11.098929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:11.098768    2566 scope.go:117] "RemoveContainer" containerID="36e195f02a3628b7e987dbafa0c529a101a0a401b9de1cdfd14495ac0ee52847"
Jul 17 11:34:11.099049 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:34:11.098983    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv_odh-ai-gateway-infra(2cb83089-7468-4859-8c49-e09ef61261dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" podUID="2cb83089-7468-4859-8c49-e09ef61261dc"
Jul 17 11:34:12.107271 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.107237    2566 generic.go:356] "Generic (PLEG): container finished" podID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerID="d99057f4a0f134b00aa206992e4e78f7183f78e243cbf436d7c311fa5057f328" exitCode=0
Jul 17 11:34:12.107745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.107314    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" event={"ID":"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c","Type":"ContainerDied","Data":"d99057f4a0f134b00aa206992e4e78f7183f78e243cbf436d7c311fa5057f328"}
Jul 17 11:34:12.107745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.107371    2566 scope.go:117] "RemoveContainer" containerID="f848e79ca95ad41e9aa54e867c80ad000ee50f2b9bba65bcd98fab0b929834cc"
Jul 17 11:34:12.109132 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.109115    2566 scope.go:117] "RemoveContainer" containerID="36e195f02a3628b7e987dbafa0c529a101a0a401b9de1cdfd14495ac0ee52847"
Jul 17 11:34:12.109382 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:34:12.109360    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv_odh-ai-gateway-infra(2cb83089-7468-4859-8c49-e09ef61261dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" podUID="2cb83089-7468-4859-8c49-e09ef61261dc"
Jul 17 11:34:12.215969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.215911    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:34:12.385014 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.384980    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s968j\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-kube-api-access-s968j\") pod \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") "
Jul 17 11:34:12.385190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385027    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-credential-socket\") pod \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") "
Jul 17 11:34:12.385190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385069    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-socket\") pod \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") "
Jul 17 11:34:12.385190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385089    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-data\") pod \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") "
Jul 17 11:34:12.385190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385129    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-envoy\") pod \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") "
Jul 17 11:34:12.385190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385168    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-certs\") pod \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") "
Jul 17 11:34:12.385495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385202    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istiod-ca-cert\") pod \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") "
Jul 17 11:34:12.385495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385253    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-podinfo\") pod \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") "
Jul 17 11:34:12.385495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385281    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-token\") pod \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\" (UID: \"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c\") "
Jul 17 11:34:12.385495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385380    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" (UID: "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:34:12.385495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385391    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" (UID: "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:34:12.385782 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385615    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:12.385782 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385639    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:12.385885 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385787    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" (UID: "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:34:12.385885 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.385846    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" (UID: "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:34:12.386154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.386129    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-data" (OuterVolumeSpecName: "istio-data") pod "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" (UID: "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:34:12.387434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.387411    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" (UID: "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:34:12.387434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.387419    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-token" (OuterVolumeSpecName: "istio-token") pod "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" (UID: "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:34:12.387587 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.387413    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" (UID: "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:34:12.387587 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.387438    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-kube-api-access-s968j" (OuterVolumeSpecName: "kube-api-access-s968j") pod "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" (UID: "20d1a1a8-eeeb-4fdf-a918-57ddd018af4c"). InnerVolumeSpecName "kube-api-access-s968j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:34:12.486849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.486775    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:12.486849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.486801    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:12.486849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.486811    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:12.486849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.486820    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:12.486849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.486829    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:12.486849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.486836    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:12.486849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:12.486846    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s968j\" (UniqueName: \"kubernetes.io/projected/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c-kube-api-access-s968j\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:13.114373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:13.114345    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"
Jul 17 11:34:13.114840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:13.114386    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48" event={"ID":"20d1a1a8-eeeb-4fdf-a918-57ddd018af4c","Type":"ContainerDied","Data":"1d99df53b6190c68a5c0a2268313af2c79de17cde59256058d0b56424145ae13"}
Jul 17 11:34:13.114840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:13.114440    2566 scope.go:117] "RemoveContainer" containerID="d99057f4a0f134b00aa206992e4e78f7183f78e243cbf436d7c311fa5057f328"
Jul 17 11:34:13.137363 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:13.137340    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"]
Jul 17 11:34:13.146644 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:13.144850    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-45989be5-openshift-default-5ff9c5cdbf-zwq48"]
Jul 17 11:34:13.657677 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:13.657643    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" path="/var/lib/kubelet/pods/20d1a1a8-eeeb-4fdf-a918-57ddd018af4c/volumes"
Jul 17 11:34:19.327510 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.327476    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"]
Jul 17 11:34:19.328042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.327873    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy"
Jul 17 11:34:19.328042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.327889    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy"
Jul 17 11:34:19.328042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.327902    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy"
Jul 17 11:34:19.328042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.327907    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy"
Jul 17 11:34:19.328042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.327921    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18a031b-7580-4446-86fc-206879c312f8" containerName="revoke-keys"
Jul 17 11:34:19.328042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.327927    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18a031b-7580-4446-86fc-206879c312f8" containerName="revoke-keys"
Jul 17 11:34:19.328042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.328018    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b18a031b-7580-4446-86fc-206879c312f8" containerName="revoke-keys"
Jul 17 11:34:19.328042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.328029    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy"
Jul 17 11:34:19.328042 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.328038    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="20d1a1a8-eeeb-4fdf-a918-57ddd018af4c" containerName="istio-proxy"
Jul 17 11:34:19.332793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.332770    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.335484 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.335465    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-57741a73-openshift-default-dockercfg-8vvbs\""
Jul 17 11:34:19.343643 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.343622    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"]
Jul 17 11:34:19.445457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.445420    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-certs\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.445640 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.445468    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-podinfo\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.445640 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.445499    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-envoy\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.445640 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.445528    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-data\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.445640 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.445551    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istiod-ca-cert\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.445880 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.445665    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-socket\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.445880 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.445724    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-token\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.445880 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.445821    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4lp5\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-kube-api-access-b4lp5\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.446006 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.445897    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-credential-socket\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.546735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.546673    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-credential-socket\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.546930 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.546774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-certs\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.546930 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.546802    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-podinfo\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.546930 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.546827    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-envoy\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.546930 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.546853    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-data\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.546930 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.546879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istiod-ca-cert\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.547212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.546996    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-socket\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.547212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.547038    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-token\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.547212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.547094    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lp5\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-kube-api-access-b4lp5\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.547212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.547149    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-credential-socket\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.547212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.547174    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-certs\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.547484 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.547237    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-data\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.547484 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.547384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-socket\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.547635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.547616    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istiod-ca-cert\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.549160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.549130    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-envoy\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.549303 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.549286    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-podinfo\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.555892 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.555868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-token\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.556153 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.556133    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4lp5\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-kube-api-access-b4lp5\") pod \"e2e-derive-57741a73-openshift-default-676cd77759-vd8rc\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") " pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.644327 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.644257    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:19.782859 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.782830    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"]
Jul 17 11:34:19.784865 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:34:19.784832    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a9c4ee_27fa_4649_a9c9_209a4ffed853.slice/crio-937e5199d0585258d0a0ec9054436f1f0cf5281befda042f4454f1ec6b7a8e0d WatchSource:0}: Error finding container 937e5199d0585258d0a0ec9054436f1f0cf5281befda042f4454f1ec6b7a8e0d: Status 404 returned error can't find the container with id 937e5199d0585258d0a0ec9054436f1f0cf5281befda042f4454f1ec6b7a8e0d
Jul 17 11:34:19.786910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.786877    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:34:19.787058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.786937    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:34:19.787058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:19.786968    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:34:20.149616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:20.149541    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" event={"ID":"71a9c4ee-27fa-4649-a9c9-209a4ffed853","Type":"ContainerStarted","Data":"7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51"}
Jul 17 11:34:20.149616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:20.149595    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" event={"ID":"71a9c4ee-27fa-4649-a9c9-209a4ffed853","Type":"ContainerStarted","Data":"937e5199d0585258d0a0ec9054436f1f0cf5281befda042f4454f1ec6b7a8e0d"}
Jul 17 11:34:20.173820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:20.173773    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" podStartSLOduration=1.173757986 podStartE2EDuration="1.173757986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:34:20.171809971 +0000 UTC m=+2243.028893910" watchObservedRunningTime="2026-07-17 11:34:20.173757986 +0000 UTC m=+2243.030841931"
Jul 17 11:34:20.645385 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:20.645351    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:20.650413 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:20.650386    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:21.155399 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:21.155369    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:21.156305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:21.156282    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:24.651945 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:24.651908    2566 scope.go:117] "RemoveContainer" containerID="36e195f02a3628b7e987dbafa0c529a101a0a401b9de1cdfd14495ac0ee52847"
Jul 17 11:34:24.937932 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:24.937854    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"]
Jul 17 11:34:24.941212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:24.941196    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"
Jul 17 11:34:24.948282 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:24.948220    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"]
Jul 17 11:34:25.097784 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:25.097748    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a165ff06-b5de-4768-b03c-6f879f65293d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-vzbl4\" (UID: \"a165ff06-b5de-4768-b03c-6f879f65293d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"
Jul 17 11:34:25.175654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:25.175622    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" event={"ID":"2cb83089-7468-4859-8c49-e09ef61261dc","Type":"ContainerStarted","Data":"70db4e3e86c7eef9bf7ab739ced169539c48a3d43cd49e6b3dd5a2fb246cb126"}
Jul 17 11:34:25.198522 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:25.198446    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a165ff06-b5de-4768-b03c-6f879f65293d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-vzbl4\" (UID: \"a165ff06-b5de-4768-b03c-6f879f65293d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"
Jul 17 11:34:25.199091 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:25.199073    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a165ff06-b5de-4768-b03c-6f879f65293d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-vzbl4\" (UID: \"a165ff06-b5de-4768-b03c-6f879f65293d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"
Jul 17 11:34:25.253051 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:25.253024    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"
Jul 17 11:34:25.546404 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:25.546314    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:34:25.588039 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:25.588015    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"]
Jul 17 11:34:25.589858 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:34:25.589832    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda165ff06_b5de_4768_b03c_6f879f65293d.slice/crio-28170b441b49e0d2e0e97a9104512c403aa949758f29b9597e7f63dfcd474066 WatchSource:0}: Error finding container 28170b441b49e0d2e0e97a9104512c403aa949758f29b9597e7f63dfcd474066: Status 404 returned error can't find the container with id 28170b441b49e0d2e0e97a9104512c403aa949758f29b9597e7f63dfcd474066
Jul 17 11:34:26.181754 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:26.181640    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" event={"ID":"a165ff06-b5de-4768-b03c-6f879f65293d","Type":"ContainerStarted","Data":"e0d8585b4f3d010e2aa93fa4e8902df8c4f1df6343255c8fe621d29856b82aa2"}
Jul 17 11:34:26.181754 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:26.181698    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" event={"ID":"a165ff06-b5de-4768-b03c-6f879f65293d","Type":"ContainerStarted","Data":"28170b441b49e0d2e0e97a9104512c403aa949758f29b9597e7f63dfcd474066"}
Jul 17 11:34:26.183438 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:26.183412    2566 generic.go:356] "Generic (PLEG): container finished" podID="2cb83089-7468-4859-8c49-e09ef61261dc" containerID="70db4e3e86c7eef9bf7ab739ced169539c48a3d43cd49e6b3dd5a2fb246cb126" exitCode=6
Jul 17 11:34:26.183526 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:26.183459    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" event={"ID":"2cb83089-7468-4859-8c49-e09ef61261dc","Type":"ContainerDied","Data":"70db4e3e86c7eef9bf7ab739ced169539c48a3d43cd49e6b3dd5a2fb246cb126"}
Jul 17 11:34:26.183526 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:26.183501    2566 scope.go:117] "RemoveContainer" containerID="36e195f02a3628b7e987dbafa0c529a101a0a401b9de1cdfd14495ac0ee52847"
Jul 17 11:34:26.183731 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:26.183698    2566 scope.go:117] "RemoveContainer" containerID="70db4e3e86c7eef9bf7ab739ced169539c48a3d43cd49e6b3dd5a2fb246cb126"
Jul 17 11:34:26.183939 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:34:26.183918    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv_odh-ai-gateway-infra(2cb83089-7468-4859-8c49-e09ef61261dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" podUID="2cb83089-7468-4859-8c49-e09ef61261dc"
Jul 17 11:34:26.199634 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:26.199596    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" podStartSLOduration=2.199582978 podStartE2EDuration="2.199582978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:34:26.19849126 +0000 UTC m=+2249.055575214" watchObservedRunningTime="2026-07-17 11:34:26.199582978 +0000 UTC m=+2249.056666922"
Jul 17 11:34:26.205778 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:26.205753    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"]
Jul 17 11:34:27.190288 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:27.190252    2566 generic.go:356] "Generic (PLEG): container finished" podID="a165ff06-b5de-4768-b03c-6f879f65293d" containerID="e0d8585b4f3d010e2aa93fa4e8902df8c4f1df6343255c8fe621d29856b82aa2" exitCode=7
Jul 17 11:34:27.190746 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:27.190340    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" event={"ID":"a165ff06-b5de-4768-b03c-6f879f65293d","Type":"ContainerDied","Data":"e0d8585b4f3d010e2aa93fa4e8902df8c4f1df6343255c8fe621d29856b82aa2"}
Jul 17 11:34:27.190746 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:27.190513    2566 scope.go:117] "RemoveContainer" containerID="e0d8585b4f3d010e2aa93fa4e8902df8c4f1df6343255c8fe621d29856b82aa2"
Jul 17 11:34:27.363468 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:27.363441    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"
Jul 17 11:34:27.418314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:27.418286    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb83089-7468-4859-8c49-e09ef61261dc-maas-api-service-ca\") pod \"2cb83089-7468-4859-8c49-e09ef61261dc\" (UID: \"2cb83089-7468-4859-8c49-e09ef61261dc\") "
Jul 17 11:34:27.418647 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:27.418620    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cb83089-7468-4859-8c49-e09ef61261dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2cb83089-7468-4859-8c49-e09ef61261dc" (UID: "2cb83089-7468-4859-8c49-e09ef61261dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:34:27.518949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:27.518923    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb83089-7468-4859-8c49-e09ef61261dc-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:28.197728 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.197681    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"
Jul 17 11:34:28.197728 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.197687    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv" event={"ID":"2cb83089-7468-4859-8c49-e09ef61261dc","Type":"ContainerDied","Data":"43f68a47728bb9b09226cd53955bdca8e3bacaaaa1c3a26f142779bcfe2e89b4"}
Jul 17 11:34:28.198241 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.197767    2566 scope.go:117] "RemoveContainer" containerID="70db4e3e86c7eef9bf7ab739ced169539c48a3d43cd49e6b3dd5a2fb246cb126"
Jul 17 11:34:28.199976 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.199887    2566 generic.go:356] "Generic (PLEG): container finished" podID="a165ff06-b5de-4768-b03c-6f879f65293d" containerID="e1cb34ee56a2547cd3a9b5dce6c7e4b9f40e085ff321c0031a0b8bd051118ec2" exitCode=7
Jul 17 11:34:28.199976 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.199927    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" event={"ID":"a165ff06-b5de-4768-b03c-6f879f65293d","Type":"ContainerDied","Data":"e1cb34ee56a2547cd3a9b5dce6c7e4b9f40e085ff321c0031a0b8bd051118ec2"}
Jul 17 11:34:28.200284 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.200267    2566 scope.go:117] "RemoveContainer" containerID="e1cb34ee56a2547cd3a9b5dce6c7e4b9f40e085ff321c0031a0b8bd051118ec2"
Jul 17 11:34:28.200522 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:34:28.200499    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-vzbl4_odh-ai-gateway-infra(a165ff06-b5de-4768-b03c-6f879f65293d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" podUID="a165ff06-b5de-4768-b03c-6f879f65293d"
Jul 17 11:34:28.206562 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.206543    2566 scope.go:117] "RemoveContainer" containerID="e0d8585b4f3d010e2aa93fa4e8902df8c4f1df6343255c8fe621d29856b82aa2"
Jul 17 11:34:28.212299 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.212277    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"]
Jul 17 11:34:28.212632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.212619    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" containerName="revoke-keys"
Jul 17 11:34:28.212632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.212632    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" containerName="revoke-keys"
Jul 17 11:34:28.212756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.212645    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" containerName="revoke-keys"
Jul 17 11:34:28.212756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.212651    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" containerName="revoke-keys"
Jul 17 11:34:28.212756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.212663    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" containerName="revoke-keys"
Jul 17 11:34:28.212756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.212669    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" containerName="revoke-keys"
Jul 17 11:34:28.212756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.212736    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" containerName="revoke-keys"
Jul 17 11:34:28.212756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.212748    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" containerName="revoke-keys"
Jul 17 11:34:28.215335 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.215297    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"
Jul 17 11:34:28.225124 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.225092    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e401ec2-e63c-4b03-98b7-429e5b667716-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-66v2d\" (UID: \"7e401ec2-e63c-4b03-98b7-429e5b667716\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"
Jul 17 11:34:28.241087 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.227630    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"]
Jul 17 11:34:28.265289 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.265233    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"]
Jul 17 11:34:28.268077 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.268046    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rg7hv"]
Jul 17 11:34:28.326270 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.326224    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e401ec2-e63c-4b03-98b7-429e5b667716-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-66v2d\" (UID: \"7e401ec2-e63c-4b03-98b7-429e5b667716\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"
Jul 17 11:34:28.347730 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.326943    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e401ec2-e63c-4b03-98b7-429e5b667716-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-66v2d\" (UID: \"7e401ec2-e63c-4b03-98b7-429e5b667716\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"
Jul 17 11:34:28.543929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.543821    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"
Jul 17 11:34:28.698060 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:28.698035    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"]
Jul 17 11:34:28.699906 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:34:28.699879    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e401ec2_e63c_4b03_98b7_429e5b667716.slice/crio-7923f1330638d25424481efee0c8273f3cbbc77fba11a2fa8826fc2caa4ad85a WatchSource:0}: Error finding container 7923f1330638d25424481efee0c8273f3cbbc77fba11a2fa8826fc2caa4ad85a: Status 404 returned error can't find the container with id 7923f1330638d25424481efee0c8273f3cbbc77fba11a2fa8826fc2caa4ad85a
Jul 17 11:34:29.206942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:29.206837    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" event={"ID":"7e401ec2-e63c-4b03-98b7-429e5b667716","Type":"ContainerStarted","Data":"68eb5725ef280a1ed9aacd4c99881e57d41aadc553e52f546f3e699e6a53cac9"}
Jul 17 11:34:29.206942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:29.206887    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" event={"ID":"7e401ec2-e63c-4b03-98b7-429e5b667716","Type":"ContainerStarted","Data":"7923f1330638d25424481efee0c8273f3cbbc77fba11a2fa8826fc2caa4ad85a"}
Jul 17 11:34:29.208659 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:29.208637    2566 scope.go:117] "RemoveContainer" containerID="e1cb34ee56a2547cd3a9b5dce6c7e4b9f40e085ff321c0031a0b8bd051118ec2"
Jul 17 11:34:29.208889 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:34:29.208867    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-vzbl4_odh-ai-gateway-infra(a165ff06-b5de-4768-b03c-6f879f65293d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" podUID="a165ff06-b5de-4768-b03c-6f879f65293d"
Jul 17 11:34:29.224022 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:29.223977    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" podStartSLOduration=1.223964662 podStartE2EDuration="1.223964662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:34:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:34:29.223521314 +0000 UTC m=+2252.080605262" watchObservedRunningTime="2026-07-17 11:34:29.223964662 +0000 UTC m=+2252.081048607"
Jul 17 11:34:29.660126 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:29.660100    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" path="/var/lib/kubelet/pods/2cb83089-7468-4859-8c49-e09ef61261dc/volumes"
Jul 17 11:34:30.215990 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:30.215953    2566 generic.go:356] "Generic (PLEG): container finished" podID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerID="68eb5725ef280a1ed9aacd4c99881e57d41aadc553e52f546f3e699e6a53cac9" exitCode=6
Jul 17 11:34:30.216454 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:30.216036    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" event={"ID":"7e401ec2-e63c-4b03-98b7-429e5b667716","Type":"ContainerDied","Data":"68eb5725ef280a1ed9aacd4c99881e57d41aadc553e52f546f3e699e6a53cac9"}
Jul 17 11:34:30.216454 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:30.216233    2566 scope.go:117] "RemoveContainer" containerID="68eb5725ef280a1ed9aacd4c99881e57d41aadc553e52f546f3e699e6a53cac9"
Jul 17 11:34:31.222447 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:31.222367    2566 generic.go:356] "Generic (PLEG): container finished" podID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerID="4feeb2fcec27bc6fbc852b144bd5ec94f3a7228e91b98d38227d99a7b48f8a83" exitCode=6
Jul 17 11:34:31.222833 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:31.222447    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" event={"ID":"7e401ec2-e63c-4b03-98b7-429e5b667716","Type":"ContainerDied","Data":"4feeb2fcec27bc6fbc852b144bd5ec94f3a7228e91b98d38227d99a7b48f8a83"}
Jul 17 11:34:31.222833 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:31.222514    2566 scope.go:117] "RemoveContainer" containerID="68eb5725ef280a1ed9aacd4c99881e57d41aadc553e52f546f3e699e6a53cac9"
Jul 17 11:34:31.222833 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:31.222774    2566 scope.go:117] "RemoveContainer" containerID="4feeb2fcec27bc6fbc852b144bd5ec94f3a7228e91b98d38227d99a7b48f8a83"
Jul 17 11:34:31.223013 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:34:31.222994    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-66v2d_odh-ai-gateway-infra(7e401ec2-e63c-4b03-98b7-429e5b667716)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716"
Jul 17 11:34:32.230815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:32.230787    2566 scope.go:117] "RemoveContainer" containerID="4feeb2fcec27bc6fbc852b144bd5ec94f3a7228e91b98d38227d99a7b48f8a83"
Jul 17 11:34:32.231175 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:34:32.231025    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-66v2d_odh-ai-gateway-infra(7e401ec2-e63c-4b03-98b7-429e5b667716)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716"
Jul 17 11:34:34.242032 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:34.241996    2566 generic.go:356] "Generic (PLEG): container finished" podID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerID="7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51" exitCode=0
Jul 17 11:34:34.242482 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:34.242059    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" event={"ID":"71a9c4ee-27fa-4649-a9c9-209a4ffed853","Type":"ContainerDied","Data":"7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51"}
Jul 17 11:34:34.242482 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:34.242473    2566 scope.go:117] "RemoveContainer" containerID="7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51"
Jul 17 11:34:34.243471 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:34.243443    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:34:34.243527 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:34.243510    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:34:34.243563 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:34.243541    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:34:35.258572 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:35.258536    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" event={"ID":"71a9c4ee-27fa-4649-a9c9-209a4ffed853","Type":"ContainerStarted","Data":"125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c"}
Jul 17 11:34:35.645469 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:35.645431    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:35.646802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:35.646765    2566 patch_prober.go:28] interesting pod/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 17 11:34:35.646949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:35.646827    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 17 11:34:36.156075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:36.156040    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:36.644959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:36.644921    2566 patch_prober.go:28] interesting pod/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 17 11:34:36.645474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:36.644992    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 17 11:34:37.644769 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:37.644729    2566 patch_prober.go:28] interesting pod/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 17 11:34:37.645059 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:37.645028    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 17 11:34:38.645474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:38.645034    2566 patch_prober.go:28] interesting pod/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 17 11:34:38.645474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:38.645103    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 17 11:34:39.648397 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:39.648362    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:39.649324 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:39.649302    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:34:40.651964 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:40.651935    2566 scope.go:117] "RemoveContainer" containerID="e1cb34ee56a2547cd3a9b5dce6c7e4b9f40e085ff321c0031a0b8bd051118ec2"
Jul 17 11:34:41.289667 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:41.289641    2566 generic.go:356] "Generic (PLEG): container finished" podID="a165ff06-b5de-4768-b03c-6f879f65293d" containerID="7475c5739c3963a793f0b2ac33b76f88533e4a2df22ecb81a4efdcdbf6e56226" exitCode=7
Jul 17 11:34:41.289793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:41.289720    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" event={"ID":"a165ff06-b5de-4768-b03c-6f879f65293d","Type":"ContainerDied","Data":"7475c5739c3963a793f0b2ac33b76f88533e4a2df22ecb81a4efdcdbf6e56226"}
Jul 17 11:34:41.289842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:41.289806    2566 scope.go:117] "RemoveContainer" containerID="e1cb34ee56a2547cd3a9b5dce6c7e4b9f40e085ff321c0031a0b8bd051118ec2"
Jul 17 11:34:41.290108 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:41.290087    2566 scope.go:117] "RemoveContainer" containerID="7475c5739c3963a793f0b2ac33b76f88533e4a2df22ecb81a4efdcdbf6e56226"
Jul 17 11:34:41.290309 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:34:41.290291    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-vzbl4_odh-ai-gateway-infra(a165ff06-b5de-4768-b03c-6f879f65293d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" podUID="a165ff06-b5de-4768-b03c-6f879f65293d"
Jul 17 11:34:42.317723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:42.317671    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"]
Jul 17 11:34:42.452947 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:42.452923    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"
Jul 17 11:34:42.576957 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:42.576933    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a165ff06-b5de-4768-b03c-6f879f65293d-maas-api-service-ca\") pod \"a165ff06-b5de-4768-b03c-6f879f65293d\" (UID: \"a165ff06-b5de-4768-b03c-6f879f65293d\") "
Jul 17 11:34:42.577269 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:42.577246    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a165ff06-b5de-4768-b03c-6f879f65293d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a165ff06-b5de-4768-b03c-6f879f65293d" (UID: "a165ff06-b5de-4768-b03c-6f879f65293d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:34:42.651598 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:42.651571    2566 scope.go:117] "RemoveContainer" containerID="4feeb2fcec27bc6fbc852b144bd5ec94f3a7228e91b98d38227d99a7b48f8a83"
Jul 17 11:34:42.677981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:42.677956    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a165ff06-b5de-4768-b03c-6f879f65293d-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:43.303566 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.303539    2566 generic.go:356] "Generic (PLEG): container finished" podID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerID="e46a3976242ae204fd4585ce7275b2d29673264e879fa7d4bc968335dcd247c4" exitCode=6
Jul 17 11:34:43.303694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.303616    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" event={"ID":"7e401ec2-e63c-4b03-98b7-429e5b667716","Type":"ContainerDied","Data":"e46a3976242ae204fd4585ce7275b2d29673264e879fa7d4bc968335dcd247c4"}
Jul 17 11:34:43.303694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.303687    2566 scope.go:117] "RemoveContainer" containerID="4feeb2fcec27bc6fbc852b144bd5ec94f3a7228e91b98d38227d99a7b48f8a83"
Jul 17 11:34:43.303997 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.303977    2566 scope.go:117] "RemoveContainer" containerID="e46a3976242ae204fd4585ce7275b2d29673264e879fa7d4bc968335dcd247c4"
Jul 17 11:34:43.304228 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:34:43.304209    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-66v2d_odh-ai-gateway-infra(7e401ec2-e63c-4b03-98b7-429e5b667716)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716"
Jul 17 11:34:43.304879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.304862    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"
Jul 17 11:34:43.304955 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.304894    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4" event={"ID":"a165ff06-b5de-4768-b03c-6f879f65293d","Type":"ContainerDied","Data":"28170b441b49e0d2e0e97a9104512c403aa949758f29b9597e7f63dfcd474066"}
Jul 17 11:34:43.314595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.314581    2566 scope.go:117] "RemoveContainer" containerID="7475c5739c3963a793f0b2ac33b76f88533e4a2df22ecb81a4efdcdbf6e56226"
Jul 17 11:34:43.337875 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.337848    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"]
Jul 17 11:34:43.342160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.342141    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-vzbl4"]
Jul 17 11:34:43.657874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:43.657839    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" path="/var/lib/kubelet/pods/a165ff06-b5de-4768-b03c-6f879f65293d/volumes"
Jul 17 11:34:44.330331 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:44.330299    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"]
Jul 17 11:34:44.462453 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:44.462432    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"
Jul 17 11:34:44.595448 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:44.595370    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e401ec2-e63c-4b03-98b7-429e5b667716-maas-api-service-ca\") pod \"7e401ec2-e63c-4b03-98b7-429e5b667716\" (UID: \"7e401ec2-e63c-4b03-98b7-429e5b667716\") "
Jul 17 11:34:44.595718 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:44.595687    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e401ec2-e63c-4b03-98b7-429e5b667716-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7e401ec2-e63c-4b03-98b7-429e5b667716" (UID: "7e401ec2-e63c-4b03-98b7-429e5b667716"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:34:44.696870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:44.696839    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e401ec2-e63c-4b03-98b7-429e5b667716-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:34:45.324953 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:45.324918    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d" event={"ID":"7e401ec2-e63c-4b03-98b7-429e5b667716","Type":"ContainerDied","Data":"7923f1330638d25424481efee0c8273f3cbbc77fba11a2fa8826fc2caa4ad85a"}
Jul 17 11:34:45.324953 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:45.324941    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"
Jul 17 11:34:45.325217 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:45.325000    2566 scope.go:117] "RemoveContainer" containerID="e46a3976242ae204fd4585ce7275b2d29673264e879fa7d4bc968335dcd247c4"
Jul 17 11:34:45.350481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:45.350452    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"]
Jul 17 11:34:45.356527 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:45.356505    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-66v2d"]
Jul 17 11:34:45.657409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:34:45.657332    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" path="/var/lib/kubelet/pods/7e401ec2-e63c-4b03-98b7-429e5b667716/volumes"
Jul 17 11:35:08.201435 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201401    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"]
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201766    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201780    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201797    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201802    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201807    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201813    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201821    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201826    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201838    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201843    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201901    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb83089-7468-4859-8c49-e09ef61261dc" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201908    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201915    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201920    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201925    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" containerName="revoke-keys"
Jul 17 11:35:08.202058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.201932    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" containerName="revoke-keys"
Jul 17 11:35:08.204901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.204885    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"
Jul 17 11:35:08.207823 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.207795    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:35:08.207823 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.207818    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:35:08.211881 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.211856    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"]
Jul 17 11:35:08.296410 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.296377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b6451c-6c32-48d0-a237-06bdabffbb63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds\" (UID: \"f7b6451c-6c32-48d0-a237-06bdabffbb63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"
Jul 17 11:35:08.397566 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.397529    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b6451c-6c32-48d0-a237-06bdabffbb63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds\" (UID: \"f7b6451c-6c32-48d0-a237-06bdabffbb63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"
Jul 17 11:35:08.398130 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.398112    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b6451c-6c32-48d0-a237-06bdabffbb63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds\" (UID: \"f7b6451c-6c32-48d0-a237-06bdabffbb63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"
Jul 17 11:35:08.515983 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.515909    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"
Jul 17 11:35:08.857612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:08.857529    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"]
Jul 17 11:35:08.859868 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:35:08.859842    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b6451c_6c32_48d0_a237_06bdabffbb63.slice/crio-9e882955f883ef1faff5fb8bf484c0554c24cfe91eed5673a6a3108d79cf4461 WatchSource:0}: Error finding container 9e882955f883ef1faff5fb8bf484c0554c24cfe91eed5673a6a3108d79cf4461: Status 404 returned error can't find the container with id 9e882955f883ef1faff5fb8bf484c0554c24cfe91eed5673a6a3108d79cf4461
Jul 17 11:35:09.444729 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:09.444627    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" event={"ID":"f7b6451c-6c32-48d0-a237-06bdabffbb63","Type":"ContainerStarted","Data":"98c770266d16a6d9c90d8e627c018da5c5e8be047ca84d22a9dd5b8d3f644fff"}
Jul 17 11:35:09.444729 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:09.444678    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" event={"ID":"f7b6451c-6c32-48d0-a237-06bdabffbb63","Type":"ContainerStarted","Data":"9e882955f883ef1faff5fb8bf484c0554c24cfe91eed5673a6a3108d79cf4461"}
Jul 17 11:35:09.462154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:09.462111    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" podStartSLOduration=1.462097392 podStartE2EDuration="1.462097392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:35:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:35:09.461104272 +0000 UTC m=+2292.318188217" watchObservedRunningTime="2026-07-17 11:35:09.462097392 +0000 UTC m=+2292.319181336"
Jul 17 11:35:10.452071 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:10.452040    2566 generic.go:356] "Generic (PLEG): container finished" podID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerID="98c770266d16a6d9c90d8e627c018da5c5e8be047ca84d22a9dd5b8d3f644fff" exitCode=6
Jul 17 11:35:10.452443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:10.452077    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" event={"ID":"f7b6451c-6c32-48d0-a237-06bdabffbb63","Type":"ContainerDied","Data":"98c770266d16a6d9c90d8e627c018da5c5e8be047ca84d22a9dd5b8d3f644fff"}
Jul 17 11:35:10.452443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:10.452260    2566 scope.go:117] "RemoveContainer" containerID="98c770266d16a6d9c90d8e627c018da5c5e8be047ca84d22a9dd5b8d3f644fff"
Jul 17 11:35:11.458508 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:11.458473    2566 generic.go:356] "Generic (PLEG): container finished" podID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerID="6f05511ae67182749be78c3d5906cd2a312c0fef12f85285414b454ec4d9bc6a" exitCode=6
Jul 17 11:35:11.458961 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:11.458527    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" event={"ID":"f7b6451c-6c32-48d0-a237-06bdabffbb63","Type":"ContainerDied","Data":"6f05511ae67182749be78c3d5906cd2a312c0fef12f85285414b454ec4d9bc6a"}
Jul 17 11:35:11.458961 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:11.458575    2566 scope.go:117] "RemoveContainer" containerID="98c770266d16a6d9c90d8e627c018da5c5e8be047ca84d22a9dd5b8d3f644fff"
Jul 17 11:35:11.458961 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:11.458781    2566 scope.go:117] "RemoveContainer" containerID="6f05511ae67182749be78c3d5906cd2a312c0fef12f85285414b454ec4d9bc6a"
Jul 17 11:35:11.459089 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:35:11.458997    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds_odh-ai-gateway-infra(f7b6451c-6c32-48d0-a237-06bdabffbb63)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63"
Jul 17 11:35:12.465726 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:12.465681    2566 scope.go:117] "RemoveContainer" containerID="6f05511ae67182749be78c3d5906cd2a312c0fef12f85285414b454ec4d9bc6a"
Jul 17 11:35:12.466119 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:35:12.465881    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds_odh-ai-gateway-infra(f7b6451c-6c32-48d0-a237-06bdabffbb63)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63"
Jul 17 11:35:17.084474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.084432    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"]
Jul 17 11:35:17.085009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.084989    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" containerName="revoke-keys"
Jul 17 11:35:17.085104 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.085012    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a165ff06-b5de-4768-b03c-6f879f65293d" containerName="revoke-keys"
Jul 17 11:35:17.085160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.085111    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e401ec2-e63c-4b03-98b7-429e5b667716" containerName="revoke-keys"
Jul 17 11:35:17.089417 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.089396    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"
Jul 17 11:35:17.094992 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.094969    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"]
Jul 17 11:35:17.164502 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.164475    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843dedf-fd3e-4706-8afd-0a22c34f9c8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-wl6gk\" (UID: \"3843dedf-fd3e-4706-8afd-0a22c34f9c8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"
Jul 17 11:35:17.265064 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.265025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843dedf-fd3e-4706-8afd-0a22c34f9c8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-wl6gk\" (UID: \"3843dedf-fd3e-4706-8afd-0a22c34f9c8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"
Jul 17 11:35:17.265666 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.265647    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843dedf-fd3e-4706-8afd-0a22c34f9c8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-wl6gk\" (UID: \"3843dedf-fd3e-4706-8afd-0a22c34f9c8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"
Jul 17 11:35:17.401682 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.401600    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"
Jul 17 11:35:17.738517 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:17.738472    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"]
Jul 17 11:35:17.740757 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:35:17.740726    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3843dedf_fd3e_4706_8afd_0a22c34f9c8b.slice/crio-85b3081fcd397c10801ff9979cd05a5a9165f124e795647a4e2fe5f7e7fbf029 WatchSource:0}: Error finding container 85b3081fcd397c10801ff9979cd05a5a9165f124e795647a4e2fe5f7e7fbf029: Status 404 returned error can't find the container with id 85b3081fcd397c10801ff9979cd05a5a9165f124e795647a4e2fe5f7e7fbf029
Jul 17 11:35:18.256328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.256295    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"]
Jul 17 11:35:18.260161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.260141    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"
Jul 17 11:35:18.266877 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.266848    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"]
Jul 17 11:35:18.275463 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.275395    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86cbd924-82ea-4e1b-be40-2e34cb94d401-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt\" (UID: \"86cbd924-82ea-4e1b-be40-2e34cb94d401\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"
Jul 17 11:35:18.381535 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.381505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86cbd924-82ea-4e1b-be40-2e34cb94d401-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt\" (UID: \"86cbd924-82ea-4e1b-be40-2e34cb94d401\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"
Jul 17 11:35:18.382160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.382138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86cbd924-82ea-4e1b-be40-2e34cb94d401-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt\" (UID: \"86cbd924-82ea-4e1b-be40-2e34cb94d401\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"
Jul 17 11:35:18.499046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.499012    2566 generic.go:356] "Generic (PLEG): container finished" podID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerID="b94fc1ee0e37c074d8cad32550a1f376ea15149360af6c06d5f6c7d6ee63ea97" exitCode=6
Jul 17 11:35:18.499213 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.499110    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk" event={"ID":"3843dedf-fd3e-4706-8afd-0a22c34f9c8b","Type":"ContainerDied","Data":"b94fc1ee0e37c074d8cad32550a1f376ea15149360af6c06d5f6c7d6ee63ea97"}
Jul 17 11:35:18.499258 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.499209    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk" event={"ID":"3843dedf-fd3e-4706-8afd-0a22c34f9c8b","Type":"ContainerStarted","Data":"85b3081fcd397c10801ff9979cd05a5a9165f124e795647a4e2fe5f7e7fbf029"}
Jul 17 11:35:18.499313 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.499256    2566 scope.go:117] "RemoveContainer" containerID="b94fc1ee0e37c074d8cad32550a1f376ea15149360af6c06d5f6c7d6ee63ea97"
Jul 17 11:35:18.571821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.571797    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"
Jul 17 11:35:18.701422 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:18.701398    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"]
Jul 17 11:35:18.702790 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:35:18.702765    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86cbd924_82ea_4e1b_be40_2e34cb94d401.slice/crio-6fe89906d54b1e15b8f6db3aa4bf6bc2c0afe9d944e998bbd7773bb29ea5d27e WatchSource:0}: Error finding container 6fe89906d54b1e15b8f6db3aa4bf6bc2c0afe9d944e998bbd7773bb29ea5d27e: Status 404 returned error can't find the container with id 6fe89906d54b1e15b8f6db3aa4bf6bc2c0afe9d944e998bbd7773bb29ea5d27e
Jul 17 11:35:19.505898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:19.505814    2566 generic.go:356] "Generic (PLEG): container finished" podID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerID="c77e41d366d2ad8b1585a94567effb1de9d6237e163783354e70d239b8ba86da" exitCode=6
Jul 17 11:35:19.506314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:19.505905    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk" event={"ID":"3843dedf-fd3e-4706-8afd-0a22c34f9c8b","Type":"ContainerDied","Data":"c77e41d366d2ad8b1585a94567effb1de9d6237e163783354e70d239b8ba86da"}
Jul 17 11:35:19.506314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:19.505978    2566 scope.go:117] "RemoveContainer" containerID="b94fc1ee0e37c074d8cad32550a1f376ea15149360af6c06d5f6c7d6ee63ea97"
Jul 17 11:35:19.506314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:19.506212    2566 scope.go:117] "RemoveContainer" containerID="c77e41d366d2ad8b1585a94567effb1de9d6237e163783354e70d239b8ba86da"
Jul 17 11:35:19.506495 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:35:19.506457    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-wl6gk_odh-ai-gateway-infra(3843dedf-fd3e-4706-8afd-0a22c34f9c8b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b"
Jul 17 11:35:19.507697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:19.507669    2566 generic.go:356] "Generic (PLEG): container finished" podID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerID="8723f622129145c807e1a08959421babf53857d71a0ddca0131a21eef75a9100" exitCode=6
Jul 17 11:35:19.507830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:19.507760    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt" event={"ID":"86cbd924-82ea-4e1b-be40-2e34cb94d401","Type":"ContainerDied","Data":"8723f622129145c807e1a08959421babf53857d71a0ddca0131a21eef75a9100"}
Jul 17 11:35:19.507885 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:19.507839    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt" event={"ID":"86cbd924-82ea-4e1b-be40-2e34cb94d401","Type":"ContainerStarted","Data":"6fe89906d54b1e15b8f6db3aa4bf6bc2c0afe9d944e998bbd7773bb29ea5d27e"}
Jul 17 11:35:19.507935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:19.507921    2566 scope.go:117] "RemoveContainer" containerID="8723f622129145c807e1a08959421babf53857d71a0ddca0131a21eef75a9100"
Jul 17 11:35:20.514584 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:20.514490    2566 generic.go:356] "Generic (PLEG): container finished" podID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerID="6a064c3a4455874556a3c7128c10c7864c58917b1f51d7734b861b1e08049cab" exitCode=6
Jul 17 11:35:20.515062 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:20.514582    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt" event={"ID":"86cbd924-82ea-4e1b-be40-2e34cb94d401","Type":"ContainerDied","Data":"6a064c3a4455874556a3c7128c10c7864c58917b1f51d7734b861b1e08049cab"}
Jul 17 11:35:20.515062 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:20.514653    2566 scope.go:117] "RemoveContainer" containerID="8723f622129145c807e1a08959421babf53857d71a0ddca0131a21eef75a9100"
Jul 17 11:35:20.515062 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:20.514867    2566 scope.go:117] "RemoveContainer" containerID="6a064c3a4455874556a3c7128c10c7864c58917b1f51d7734b861b1e08049cab"
Jul 17 11:35:20.515203 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:35:20.515103    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt_odh-ai-gateway-infra(86cbd924-82ea-4e1b-be40-2e34cb94d401)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401"
Jul 17 11:35:20.516530 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:20.516513    2566 scope.go:117] "RemoveContainer" containerID="c77e41d366d2ad8b1585a94567effb1de9d6237e163783354e70d239b8ba86da"
Jul 17 11:35:20.516674 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:35:20.516658    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-wl6gk_odh-ai-gateway-infra(3843dedf-fd3e-4706-8afd-0a22c34f9c8b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b"
Jul 17 11:35:21.522481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:21.522454    2566 scope.go:117] "RemoveContainer" containerID="6a064c3a4455874556a3c7128c10c7864c58917b1f51d7734b861b1e08049cab"
Jul 17 11:35:21.522889 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:35:21.522645    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt_odh-ai-gateway-infra(86cbd924-82ea-4e1b-be40-2e34cb94d401)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401"
Jul 17 11:35:24.651792 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:24.651764    2566 scope.go:117] "RemoveContainer" containerID="6f05511ae67182749be78c3d5906cd2a312c0fef12f85285414b454ec4d9bc6a"
Jul 17 11:35:25.543552 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:25.543467    2566 generic.go:356] "Generic (PLEG): container finished" podID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerID="907a703a89e65221017dbfc64ef6ccdae7a81c2663d183aa0dadddbb97307d84" exitCode=6
Jul 17 11:35:25.543552 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:25.543522    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" event={"ID":"f7b6451c-6c32-48d0-a237-06bdabffbb63","Type":"ContainerDied","Data":"907a703a89e65221017dbfc64ef6ccdae7a81c2663d183aa0dadddbb97307d84"}
Jul 17 11:35:25.543832 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:25.543570    2566 scope.go:117] "RemoveContainer" containerID="6f05511ae67182749be78c3d5906cd2a312c0fef12f85285414b454ec4d9bc6a"
Jul 17 11:35:25.543929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:25.543911    2566 scope.go:117] "RemoveContainer" containerID="907a703a89e65221017dbfc64ef6ccdae7a81c2663d183aa0dadddbb97307d84"
Jul 17 11:35:25.544135 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:35:25.544113    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds_odh-ai-gateway-infra(f7b6451c-6c32-48d0-a237-06bdabffbb63)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63"
Jul 17 11:35:26.571894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:26.571860    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"]
Jul 17 11:35:26.704136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:26.704111    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"
Jul 17 11:35:26.754435 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:26.754405    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b6451c-6c32-48d0-a237-06bdabffbb63-maas-api-service-ca\") pod \"f7b6451c-6c32-48d0-a237-06bdabffbb63\" (UID: \"f7b6451c-6c32-48d0-a237-06bdabffbb63\") "
Jul 17 11:35:26.754812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:26.754792    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b6451c-6c32-48d0-a237-06bdabffbb63-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f7b6451c-6c32-48d0-a237-06bdabffbb63" (UID: "f7b6451c-6c32-48d0-a237-06bdabffbb63"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:35:26.856049 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:26.856028    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b6451c-6c32-48d0-a237-06bdabffbb63-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:35:27.554973 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:27.554944    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"
Jul 17 11:35:27.554973 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:27.554961    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds" event={"ID":"f7b6451c-6c32-48d0-a237-06bdabffbb63","Type":"ContainerDied","Data":"9e882955f883ef1faff5fb8bf484c0554c24cfe91eed5673a6a3108d79cf4461"}
Jul 17 11:35:27.555238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:27.555029    2566 scope.go:117] "RemoveContainer" containerID="907a703a89e65221017dbfc64ef6ccdae7a81c2663d183aa0dadddbb97307d84"
Jul 17 11:35:27.579800 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:27.579778    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"]
Jul 17 11:35:27.586292 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:27.586267    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dz8ds"]
Jul 17 11:35:27.657107 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:27.657085    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" path="/var/lib/kubelet/pods/f7b6451c-6c32-48d0-a237-06bdabffbb63/volumes"
Jul 17 11:35:34.652301 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:34.652269    2566 scope.go:117] "RemoveContainer" containerID="c77e41d366d2ad8b1585a94567effb1de9d6237e163783354e70d239b8ba86da"
Jul 17 11:35:35.601863 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:35.601798    2566 generic.go:356] "Generic (PLEG): container finished" podID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerID="67c52654903f1358e4ca5ddfa45583b066d3fbfa332be5c3cba986d708d8197b" exitCode=6
Jul 17 11:35:35.601990 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:35.601861    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk" event={"ID":"3843dedf-fd3e-4706-8afd-0a22c34f9c8b","Type":"ContainerDied","Data":"67c52654903f1358e4ca5ddfa45583b066d3fbfa332be5c3cba986d708d8197b"}
Jul 17 11:35:35.601990 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:35.601911    2566 scope.go:117] "RemoveContainer" containerID="c77e41d366d2ad8b1585a94567effb1de9d6237e163783354e70d239b8ba86da"
Jul 17 11:35:35.602199 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:35.602168    2566 scope.go:117] "RemoveContainer" containerID="67c52654903f1358e4ca5ddfa45583b066d3fbfa332be5c3cba986d708d8197b"
Jul 17 11:35:35.602401 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:35:35.602378    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-wl6gk_odh-ai-gateway-infra(3843dedf-fd3e-4706-8afd-0a22c34f9c8b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b"
Jul 17 11:35:36.633489 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:36.633453    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"]
Jul 17 11:35:36.651658 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:36.651629    2566 scope.go:117] "RemoveContainer" containerID="6a064c3a4455874556a3c7128c10c7864c58917b1f51d7734b861b1e08049cab"
Jul 17 11:35:36.773037 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:36.773011    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"
Jul 17 11:35:36.837937 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:36.837912    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843dedf-fd3e-4706-8afd-0a22c34f9c8b-maas-api-service-ca\") pod \"3843dedf-fd3e-4706-8afd-0a22c34f9c8b\" (UID: \"3843dedf-fd3e-4706-8afd-0a22c34f9c8b\") "
Jul 17 11:35:36.838289 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:36.838271    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3843dedf-fd3e-4706-8afd-0a22c34f9c8b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3843dedf-fd3e-4706-8afd-0a22c34f9c8b" (UID: "3843dedf-fd3e-4706-8afd-0a22c34f9c8b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:35:36.939320 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:36.939295    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843dedf-fd3e-4706-8afd-0a22c34f9c8b-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:35:37.614845 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:37.614809    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"
Jul 17 11:35:37.614845 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:37.614828    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk" event={"ID":"3843dedf-fd3e-4706-8afd-0a22c34f9c8b","Type":"ContainerDied","Data":"85b3081fcd397c10801ff9979cd05a5a9165f124e795647a4e2fe5f7e7fbf029"}
Jul 17 11:35:37.615105 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:37.614885    2566 scope.go:117] "RemoveContainer" containerID="67c52654903f1358e4ca5ddfa45583b066d3fbfa332be5c3cba986d708d8197b"
Jul 17 11:35:37.616682 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:37.616653    2566 generic.go:356] "Generic (PLEG): container finished" podID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerID="9f353b7a8d6d125811fbf57c8359751e929a1b33521a5ad19c4c4ba70dfca3d6" exitCode=6
Jul 17 11:35:37.616857 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:37.616737    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt" event={"ID":"86cbd924-82ea-4e1b-be40-2e34cb94d401","Type":"ContainerDied","Data":"9f353b7a8d6d125811fbf57c8359751e929a1b33521a5ad19c4c4ba70dfca3d6"}
Jul 17 11:35:37.617009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:37.616996    2566 scope.go:117] "RemoveContainer" containerID="9f353b7a8d6d125811fbf57c8359751e929a1b33521a5ad19c4c4ba70dfca3d6"
Jul 17 11:35:37.617183 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:35:37.617168    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt_odh-ai-gateway-infra(86cbd924-82ea-4e1b-be40-2e34cb94d401)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401"
Jul 17 11:35:37.623348 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:37.623329    2566 scope.go:117] "RemoveContainer" containerID="6a064c3a4455874556a3c7128c10c7864c58917b1f51d7734b861b1e08049cab"
Jul 17 11:35:37.657582 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:37.657564    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"]
Jul 17 11:35:37.661848 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:37.661829    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-wl6gk"]
Jul 17 11:35:38.649451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:38.649421    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"]
Jul 17 11:35:38.782260 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:38.782238    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"
Jul 17 11:35:38.857102 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:38.857076    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86cbd924-82ea-4e1b-be40-2e34cb94d401-maas-api-service-ca\") pod \"86cbd924-82ea-4e1b-be40-2e34cb94d401\" (UID: \"86cbd924-82ea-4e1b-be40-2e34cb94d401\") "
Jul 17 11:35:38.857416 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:38.857396    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86cbd924-82ea-4e1b-be40-2e34cb94d401-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "86cbd924-82ea-4e1b-be40-2e34cb94d401" (UID: "86cbd924-82ea-4e1b-be40-2e34cb94d401"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:35:38.958231 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:38.958152    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86cbd924-82ea-4e1b-be40-2e34cb94d401-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:35:39.630677 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:39.630646    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"
Jul 17 11:35:39.630863 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:39.630649    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt" event={"ID":"86cbd924-82ea-4e1b-be40-2e34cb94d401","Type":"ContainerDied","Data":"6fe89906d54b1e15b8f6db3aa4bf6bc2c0afe9d944e998bbd7773bb29ea5d27e"}
Jul 17 11:35:39.630863 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:39.630783    2566 scope.go:117] "RemoveContainer" containerID="9f353b7a8d6d125811fbf57c8359751e929a1b33521a5ad19c4c4ba70dfca3d6"
Jul 17 11:35:39.657660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:39.657632    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" path="/var/lib/kubelet/pods/3843dedf-fd3e-4706-8afd-0a22c34f9c8b/volumes"
Jul 17 11:35:39.658045 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:39.658022    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"]
Jul 17 11:35:39.660145 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:39.660126    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-z9ztt"]
Jul 17 11:35:41.658437 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:35:41.658403    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" path="/var/lib/kubelet/pods/86cbd924-82ea-4e1b-be40-2e34cb94d401/volumes"
Jul 17 11:36:15.028776 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.028742    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"]
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029093    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029103    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029113    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029119    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029128    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029133    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029141    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029148    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029156    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029162    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029169    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029174    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029189    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029194    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029254    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029263    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerName="revoke-keys"
Jul 17 11:36:15.029261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029270    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerName="revoke-keys"
Jul 17 11:36:15.030030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029275    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3843dedf-fd3e-4706-8afd-0a22c34f9c8b" containerName="revoke-keys"
Jul 17 11:36:15.030030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029281    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerName="revoke-keys"
Jul 17 11:36:15.030030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029286    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerName="revoke-keys"
Jul 17 11:36:15.030030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.029291    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerName="revoke-keys"
Jul 17 11:36:15.032296 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.032280    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"
Jul 17 11:36:15.035030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.035009    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:36:15.035104 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.035069    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:36:15.041674 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.041641    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"]
Jul 17 11:36:15.054133 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.054108    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e267d56a-9bf7-40c7-b493-34d451a8783c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-t27bm\" (UID: \"e267d56a-9bf7-40c7-b493-34d451a8783c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"
Jul 17 11:36:15.155208 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.155180    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e267d56a-9bf7-40c7-b493-34d451a8783c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-t27bm\" (UID: \"e267d56a-9bf7-40c7-b493-34d451a8783c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"
Jul 17 11:36:15.155855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.155835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e267d56a-9bf7-40c7-b493-34d451a8783c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-t27bm\" (UID: \"e267d56a-9bf7-40c7-b493-34d451a8783c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"
Jul 17 11:36:15.344516 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.344487    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"
Jul 17 11:36:15.476085 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.475901    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"]
Jul 17 11:36:15.478514 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:36:15.478486    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode267d56a_9bf7_40c7_b493_34d451a8783c.slice/crio-cffccaba18ba3abd564f0db03c28c05ef9590e4dc039fb0ff0720c196558a950 WatchSource:0}: Error finding container cffccaba18ba3abd564f0db03c28c05ef9590e4dc039fb0ff0720c196558a950: Status 404 returned error can't find the container with id cffccaba18ba3abd564f0db03c28c05ef9590e4dc039fb0ff0720c196558a950
Jul 17 11:36:15.799309 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.799261    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" event={"ID":"e267d56a-9bf7-40c7-b493-34d451a8783c","Type":"ContainerStarted","Data":"d22bb4b9cfe7cc37144a5d9a76af8eb3c519c8bc835ce16cabf78a9110c8bc9c"}
Jul 17 11:36:15.799503 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.799320    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" event={"ID":"e267d56a-9bf7-40c7-b493-34d451a8783c","Type":"ContainerStarted","Data":"cffccaba18ba3abd564f0db03c28c05ef9590e4dc039fb0ff0720c196558a950"}
Jul 17 11:36:15.816827 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:15.816784    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" podStartSLOduration=0.816769357 podStartE2EDuration="816.769357ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:36:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:36:15.815903821 +0000 UTC m=+2358.672987766" watchObservedRunningTime="2026-07-17 11:36:15.816769357 +0000 UTC m=+2358.673853301"
Jul 17 11:36:16.806782 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:16.806751    2566 generic.go:356] "Generic (PLEG): container finished" podID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerID="d22bb4b9cfe7cc37144a5d9a76af8eb3c519c8bc835ce16cabf78a9110c8bc9c" exitCode=7
Jul 17 11:36:16.807162 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:16.806836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" event={"ID":"e267d56a-9bf7-40c7-b493-34d451a8783c","Type":"ContainerDied","Data":"d22bb4b9cfe7cc37144a5d9a76af8eb3c519c8bc835ce16cabf78a9110c8bc9c"}
Jul 17 11:36:16.807162 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:16.807016    2566 scope.go:117] "RemoveContainer" containerID="d22bb4b9cfe7cc37144a5d9a76af8eb3c519c8bc835ce16cabf78a9110c8bc9c"
Jul 17 11:36:17.141154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.141129    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"]
Jul 17 11:36:17.141765 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.141738    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerName="revoke-keys"
Jul 17 11:36:17.141765 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.141757    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerName="revoke-keys"
Jul 17 11:36:17.141855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.141776    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerName="revoke-keys"
Jul 17 11:36:17.141855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.141785    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerName="revoke-keys"
Jul 17 11:36:17.141925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.141866    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="86cbd924-82ea-4e1b-be40-2e34cb94d401" containerName="revoke-keys"
Jul 17 11:36:17.141925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.141879    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b6451c-6c32-48d0-a237-06bdabffbb63" containerName="revoke-keys"
Jul 17 11:36:17.145282 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.145265    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"
Jul 17 11:36:17.153825 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.153801    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"]
Jul 17 11:36:17.171886 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.171861    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c859032-b734-41e1-b309-483f21efb93b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-c6rwl\" (UID: \"2c859032-b734-41e1-b309-483f21efb93b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"
Jul 17 11:36:17.272651 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.272613    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c859032-b734-41e1-b309-483f21efb93b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-c6rwl\" (UID: \"2c859032-b734-41e1-b309-483f21efb93b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"
Jul 17 11:36:17.273310 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.273284    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c859032-b734-41e1-b309-483f21efb93b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-c6rwl\" (UID: \"2c859032-b734-41e1-b309-483f21efb93b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"
Jul 17 11:36:17.456131 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.456050    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"
Jul 17 11:36:17.586360 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.586335    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"]
Jul 17 11:36:17.600551 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:36:17.600518    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c859032_b734_41e1_b309_483f21efb93b.slice/crio-ed4ba60fbe65084864242da0d4d43c4dc326446f4782e2da139bbb26de9d3fbc WatchSource:0}: Error finding container ed4ba60fbe65084864242da0d4d43c4dc326446f4782e2da139bbb26de9d3fbc: Status 404 returned error can't find the container with id ed4ba60fbe65084864242da0d4d43c4dc326446f4782e2da139bbb26de9d3fbc
Jul 17 11:36:17.813016 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.812937    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" event={"ID":"2c859032-b734-41e1-b309-483f21efb93b","Type":"ContainerStarted","Data":"ed4ba60fbe65084864242da0d4d43c4dc326446f4782e2da139bbb26de9d3fbc"}
Jul 17 11:36:17.814696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.814669    2566 generic.go:356] "Generic (PLEG): container finished" podID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerID="c0ab2b746d00e3a23234b0c00f4487e181876726968d25345db714234468c4f3" exitCode=7
Jul 17 11:36:17.814873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.814727    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" event={"ID":"e267d56a-9bf7-40c7-b493-34d451a8783c","Type":"ContainerDied","Data":"c0ab2b746d00e3a23234b0c00f4487e181876726968d25345db714234468c4f3"}
Jul 17 11:36:17.814873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.814769    2566 scope.go:117] "RemoveContainer" containerID="d22bb4b9cfe7cc37144a5d9a76af8eb3c519c8bc835ce16cabf78a9110c8bc9c"
Jul 17 11:36:17.815001 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:17.814948    2566 scope.go:117] "RemoveContainer" containerID="c0ab2b746d00e3a23234b0c00f4487e181876726968d25345db714234468c4f3"
Jul 17 11:36:17.815187 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:36:17.815168    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-t27bm_odh-ai-gateway-infra(e267d56a-9bf7-40c7-b493-34d451a8783c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c"
Jul 17 11:36:18.821750 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:18.821649    2566 scope.go:117] "RemoveContainer" containerID="c0ab2b746d00e3a23234b0c00f4487e181876726968d25345db714234468c4f3"
Jul 17 11:36:18.822163 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:36:18.821879    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-t27bm_odh-ai-gateway-infra(e267d56a-9bf7-40c7-b493-34d451a8783c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c"
Jul 17 11:36:18.822943 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:18.822923    2566 generic.go:356] "Generic (PLEG): container finished" podID="2c859032-b734-41e1-b309-483f21efb93b" containerID="ed78dec489aaa3d566358bfe98e2497b3fa3faf738f9cccd90638bd55f627635" exitCode=6
Jul 17 11:36:18.823026 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:18.823006    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" event={"ID":"2c859032-b734-41e1-b309-483f21efb93b","Type":"ContainerDied","Data":"ed78dec489aaa3d566358bfe98e2497b3fa3faf738f9cccd90638bd55f627635"}
Jul 17 11:36:18.823126 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:18.823114    2566 scope.go:117] "RemoveContainer" containerID="ed78dec489aaa3d566358bfe98e2497b3fa3faf738f9cccd90638bd55f627635"
Jul 17 11:36:19.830017 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:19.829981    2566 generic.go:356] "Generic (PLEG): container finished" podID="2c859032-b734-41e1-b309-483f21efb93b" containerID="52da604a04e610e39ea0b8bff314a0ace0211df53a806ddccd879f038fc5d047" exitCode=6
Jul 17 11:36:19.830420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:19.830064    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" event={"ID":"2c859032-b734-41e1-b309-483f21efb93b","Type":"ContainerDied","Data":"52da604a04e610e39ea0b8bff314a0ace0211df53a806ddccd879f038fc5d047"}
Jul 17 11:36:19.830420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:19.830121    2566 scope.go:117] "RemoveContainer" containerID="ed78dec489aaa3d566358bfe98e2497b3fa3faf738f9cccd90638bd55f627635"
Jul 17 11:36:19.830420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:19.830295    2566 scope.go:117] "RemoveContainer" containerID="52da604a04e610e39ea0b8bff314a0ace0211df53a806ddccd879f038fc5d047"
Jul 17 11:36:19.830554 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:36:19.830533    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-c6rwl_odh-ai-gateway-infra(2c859032-b734-41e1-b309-483f21efb93b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" podUID="2c859032-b734-41e1-b309-483f21efb93b"
Jul 17 11:36:20.836415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:20.836388    2566 scope.go:117] "RemoveContainer" containerID="52da604a04e610e39ea0b8bff314a0ace0211df53a806ddccd879f038fc5d047"
Jul 17 11:36:20.836849 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:36:20.836551    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-c6rwl_odh-ai-gateway-infra(2c859032-b734-41e1-b309-483f21efb93b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" podUID="2c859032-b734-41e1-b309-483f21efb93b"
Jul 17 11:36:29.652362 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:29.652334    2566 scope.go:117] "RemoveContainer" containerID="c0ab2b746d00e3a23234b0c00f4487e181876726968d25345db714234468c4f3"
Jul 17 11:36:29.880103 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:29.880078    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" event={"ID":"e267d56a-9bf7-40c7-b493-34d451a8783c","Type":"ContainerStarted","Data":"bc846ff821363b35ade8e849f5eddecce58dec812d20e2b3382d5baab9ebb3c7"}
Jul 17 11:36:30.887145 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:30.887114    2566 generic.go:356] "Generic (PLEG): container finished" podID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerID="bc846ff821363b35ade8e849f5eddecce58dec812d20e2b3382d5baab9ebb3c7" exitCode=7
Jul 17 11:36:30.887614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:30.887162    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" event={"ID":"e267d56a-9bf7-40c7-b493-34d451a8783c","Type":"ContainerDied","Data":"bc846ff821363b35ade8e849f5eddecce58dec812d20e2b3382d5baab9ebb3c7"}
Jul 17 11:36:30.887614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:30.887221    2566 scope.go:117] "RemoveContainer" containerID="c0ab2b746d00e3a23234b0c00f4487e181876726968d25345db714234468c4f3"
Jul 17 11:36:30.887614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:30.887441    2566 scope.go:117] "RemoveContainer" containerID="bc846ff821363b35ade8e849f5eddecce58dec812d20e2b3382d5baab9ebb3c7"
Jul 17 11:36:30.887783 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:36:30.887665    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-t27bm_odh-ai-gateway-infra(e267d56a-9bf7-40c7-b493-34d451a8783c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c"
Jul 17 11:36:30.918637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:30.918614    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"]
Jul 17 11:36:32.033221 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:32.033195    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"
Jul 17 11:36:32.098594 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:32.098563    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e267d56a-9bf7-40c7-b493-34d451a8783c-maas-api-service-ca\") pod \"e267d56a-9bf7-40c7-b493-34d451a8783c\" (UID: \"e267d56a-9bf7-40c7-b493-34d451a8783c\") "
Jul 17 11:36:32.098943 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:32.098919    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e267d56a-9bf7-40c7-b493-34d451a8783c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e267d56a-9bf7-40c7-b493-34d451a8783c" (UID: "e267d56a-9bf7-40c7-b493-34d451a8783c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:36:32.199341 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:32.199245    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e267d56a-9bf7-40c7-b493-34d451a8783c-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:36:32.900764 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:32.900730    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm" event={"ID":"e267d56a-9bf7-40c7-b493-34d451a8783c","Type":"ContainerDied","Data":"cffccaba18ba3abd564f0db03c28c05ef9590e4dc039fb0ff0720c196558a950"}
Jul 17 11:36:32.900764 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:32.900761    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"
Jul 17 11:36:32.900966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:32.900792    2566 scope.go:117] "RemoveContainer" containerID="bc846ff821363b35ade8e849f5eddecce58dec812d20e2b3382d5baab9ebb3c7"
Jul 17 11:36:32.924740 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:32.924687    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"]
Jul 17 11:36:32.933102 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:32.933080    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t27bm"]
Jul 17 11:36:33.657853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:33.657823    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" path="/var/lib/kubelet/pods/e267d56a-9bf7-40c7-b493-34d451a8783c/volumes"
Jul 17 11:36:34.652611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:34.652578    2566 scope.go:117] "RemoveContainer" containerID="52da604a04e610e39ea0b8bff314a0ace0211df53a806ddccd879f038fc5d047"
Jul 17 11:36:34.912821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:34.912732    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" event={"ID":"2c859032-b734-41e1-b309-483f21efb93b","Type":"ContainerStarted","Data":"2e806ced9d663a7655b525aaed0e89e48bb94e723aba47ab5581695de1bf3a35"}
Jul 17 11:36:34.932212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:34.932160    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" podStartSLOduration=17.932143475 podStartE2EDuration="17.932143475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:36:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:36:34.9313557 +0000 UTC m=+2377.788439646" watchObservedRunningTime="2026-07-17 11:36:34.932143475 +0000 UTC m=+2377.789227420"
Jul 17 11:36:35.919635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:35.919601    2566 generic.go:356] "Generic (PLEG): container finished" podID="2c859032-b734-41e1-b309-483f21efb93b" containerID="2e806ced9d663a7655b525aaed0e89e48bb94e723aba47ab5581695de1bf3a35" exitCode=6
Jul 17 11:36:35.920034 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:35.919677    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" event={"ID":"2c859032-b734-41e1-b309-483f21efb93b","Type":"ContainerDied","Data":"2e806ced9d663a7655b525aaed0e89e48bb94e723aba47ab5581695de1bf3a35"}
Jul 17 11:36:35.920034 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:35.919763    2566 scope.go:117] "RemoveContainer" containerID="52da604a04e610e39ea0b8bff314a0ace0211df53a806ddccd879f038fc5d047"
Jul 17 11:36:35.920127 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:35.920051    2566 scope.go:117] "RemoveContainer" containerID="2e806ced9d663a7655b525aaed0e89e48bb94e723aba47ab5581695de1bf3a35"
Jul 17 11:36:35.920299 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:36:35.920274    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-c6rwl_odh-ai-gateway-infra(2c859032-b734-41e1-b309-483f21efb93b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" podUID="2c859032-b734-41e1-b309-483f21efb93b"
Jul 17 11:36:35.949616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:35.949596    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"]
Jul 17 11:36:37.075199 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:37.075179    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"
Jul 17 11:36:37.145924 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:37.145900    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c859032-b734-41e1-b309-483f21efb93b-maas-api-service-ca\") pod \"2c859032-b734-41e1-b309-483f21efb93b\" (UID: \"2c859032-b734-41e1-b309-483f21efb93b\") "
Jul 17 11:36:37.146225 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:37.146206    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c859032-b734-41e1-b309-483f21efb93b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c859032-b734-41e1-b309-483f21efb93b" (UID: "2c859032-b734-41e1-b309-483f21efb93b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:36:37.247400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:37.247334    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c859032-b734-41e1-b309-483f21efb93b-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:36:37.931476 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:37.931389    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl" event={"ID":"2c859032-b734-41e1-b309-483f21efb93b","Type":"ContainerDied","Data":"ed4ba60fbe65084864242da0d4d43c4dc326446f4782e2da139bbb26de9d3fbc"}
Jul 17 11:36:37.931476 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:37.931412    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"
Jul 17 11:36:37.931476 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:37.931459    2566 scope.go:117] "RemoveContainer" containerID="2e806ced9d663a7655b525aaed0e89e48bb94e723aba47ab5581695de1bf3a35"
Jul 17 11:36:37.952838 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:37.952813    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"]
Jul 17 11:36:37.956630 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:37.956608    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-c6rwl"]
Jul 17 11:36:39.658186 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:39.658141    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c859032-b734-41e1-b309-483f21efb93b" path="/var/lib/kubelet/pods/2c859032-b734-41e1-b309-483f21efb93b/volumes"
Jul 17 11:36:57.817939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:57.817912    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:36:57.821114 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:57.821093    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:36:57.823170 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:57.823148    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:36:57.826529 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:36:57.826512    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:37:17.221720 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.221667    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"]
Jul 17 11:37:17.222110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222069    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c859032-b734-41e1-b309-483f21efb93b" containerName="revoke-keys"
Jul 17 11:37:17.222110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222082    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c859032-b734-41e1-b309-483f21efb93b" containerName="revoke-keys"
Jul 17 11:37:17.222110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222099    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerName="revoke-keys"
Jul 17 11:37:17.222110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222104    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerName="revoke-keys"
Jul 17 11:37:17.222110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222112    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c859032-b734-41e1-b309-483f21efb93b" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222118    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c859032-b734-41e1-b309-483f21efb93b" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222128    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c859032-b734-41e1-b309-483f21efb93b" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222136    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c859032-b734-41e1-b309-483f21efb93b" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222154    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222159    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222164    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222168    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222230    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222238    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222243    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c859032-b734-41e1-b309-483f21efb93b" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222249    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c859032-b734-41e1-b309-483f21efb93b" containerName="revoke-keys"
Jul 17 11:37:17.222275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.222253    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c859032-b734-41e1-b309-483f21efb93b" containerName="revoke-keys"
Jul 17 11:37:17.225201 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.225185    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"
Jul 17 11:37:17.229057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.229031    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:37:17.229690 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.229675    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:37:17.239671 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.239650    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"]
Jul 17 11:37:17.298987 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.298958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368cb778-dfc0-42fa-9c22-635c7b08dac6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-pmrqr\" (UID: \"368cb778-dfc0-42fa-9c22-635c7b08dac6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"
Jul 17 11:37:17.399413 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.399382    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368cb778-dfc0-42fa-9c22-635c7b08dac6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-pmrqr\" (UID: \"368cb778-dfc0-42fa-9c22-635c7b08dac6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"
Jul 17 11:37:17.400085 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.400061    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368cb778-dfc0-42fa-9c22-635c7b08dac6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-pmrqr\" (UID: \"368cb778-dfc0-42fa-9c22-635c7b08dac6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"
Jul 17 11:37:17.534651 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.534571    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"
Jul 17 11:37:17.667442 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:17.667411    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"]
Jul 17 11:37:17.672459 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:37:17.672429    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod368cb778_dfc0_42fa_9c22_635c7b08dac6.slice/crio-f4ad16bd9688c458e952f1dd3579cca6565c4edf819ccc3215c8e9483f86b320 WatchSource:0}: Error finding container f4ad16bd9688c458e952f1dd3579cca6565c4edf819ccc3215c8e9483f86b320: Status 404 returned error can't find the container with id f4ad16bd9688c458e952f1dd3579cca6565c4edf819ccc3215c8e9483f86b320
Jul 17 11:37:18.122431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.122351    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" event={"ID":"368cb778-dfc0-42fa-9c22-635c7b08dac6","Type":"ContainerStarted","Data":"1694570102c32064558c8ac05eb544e05f6f8888a5e76e11c7a7ca2094457ba7"}
Jul 17 11:37:18.127295 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.122806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" event={"ID":"368cb778-dfc0-42fa-9c22-635c7b08dac6","Type":"ContainerStarted","Data":"f4ad16bd9688c458e952f1dd3579cca6565c4edf819ccc3215c8e9483f86b320"}
Jul 17 11:37:18.144219 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.144170    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" podStartSLOduration=1.144157729 podStartE2EDuration="1.144157729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:37:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:37:18.142662845 +0000 UTC m=+2420.999746792" watchObservedRunningTime="2026-07-17 11:37:18.144157729 +0000 UTC m=+2421.001241686"
Jul 17 11:37:18.434050 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.433973    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"]
Jul 17 11:37:18.434401 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.434389    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e267d56a-9bf7-40c7-b493-34d451a8783c" containerName="revoke-keys"
Jul 17 11:37:18.437310 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.437292    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"
Jul 17 11:37:18.459826 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.459804    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"]
Jul 17 11:37:18.509194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.509164    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl\" (UID: \"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"
Jul 17 11:37:18.610544 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.610507    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl\" (UID: \"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"
Jul 17 11:37:18.611171 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.611152    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl\" (UID: \"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"
Jul 17 11:37:18.747196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.747119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"
Jul 17 11:37:18.889837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:18.889804    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"]
Jul 17 11:37:18.892738 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:37:18.892689    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ec4d65_ffd4_4a4b_a9b7_0f49130e6b0c.slice/crio-c009c51b24ded6c70542bc5098f889f7190ee1a82cb9912acccba8e5efab1f65 WatchSource:0}: Error finding container c009c51b24ded6c70542bc5098f889f7190ee1a82cb9912acccba8e5efab1f65: Status 404 returned error can't find the container with id c009c51b24ded6c70542bc5098f889f7190ee1a82cb9912acccba8e5efab1f65
Jul 17 11:37:19.129288 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:19.129240    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" event={"ID":"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c","Type":"ContainerStarted","Data":"c009c51b24ded6c70542bc5098f889f7190ee1a82cb9912acccba8e5efab1f65"}
Jul 17 11:37:19.131195 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:19.131168    2566 generic.go:356] "Generic (PLEG): container finished" podID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerID="1694570102c32064558c8ac05eb544e05f6f8888a5e76e11c7a7ca2094457ba7" exitCode=6
Jul 17 11:37:19.131336 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:19.131218    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" event={"ID":"368cb778-dfc0-42fa-9c22-635c7b08dac6","Type":"ContainerDied","Data":"1694570102c32064558c8ac05eb544e05f6f8888a5e76e11c7a7ca2094457ba7"}
Jul 17 11:37:19.131457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:19.131442    2566 scope.go:117] "RemoveContainer" containerID="1694570102c32064558c8ac05eb544e05f6f8888a5e76e11c7a7ca2094457ba7"
Jul 17 11:37:20.137990 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:20.137906    2566 generic.go:356] "Generic (PLEG): container finished" podID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerID="1b32269e501605d2915b30b36a0f1933214414f5cd7ae8d24fe2b88d7d6c4602" exitCode=6
Jul 17 11:37:20.138411 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:20.137993    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" event={"ID":"368cb778-dfc0-42fa-9c22-635c7b08dac6","Type":"ContainerDied","Data":"1b32269e501605d2915b30b36a0f1933214414f5cd7ae8d24fe2b88d7d6c4602"}
Jul 17 11:37:20.138411 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:20.138066    2566 scope.go:117] "RemoveContainer" containerID="1694570102c32064558c8ac05eb544e05f6f8888a5e76e11c7a7ca2094457ba7"
Jul 17 11:37:20.138411 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:20.138262    2566 scope.go:117] "RemoveContainer" containerID="1b32269e501605d2915b30b36a0f1933214414f5cd7ae8d24fe2b88d7d6c4602"
Jul 17 11:37:20.138590 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:20.138468    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-pmrqr_odh-ai-gateway-infra(368cb778-dfc0-42fa-9c22-635c7b08dac6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6"
Jul 17 11:37:20.139569 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:20.139544    2566 generic.go:356] "Generic (PLEG): container finished" podID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerID="06608e7abf32a5708a8ed210da75893502ca84a529b1fe54680e1fcdbb801e06" exitCode=6
Jul 17 11:37:20.139669 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:20.139595    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" event={"ID":"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c","Type":"ContainerDied","Data":"06608e7abf32a5708a8ed210da75893502ca84a529b1fe54680e1fcdbb801e06"}
Jul 17 11:37:20.139834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:20.139819    2566 scope.go:117] "RemoveContainer" containerID="06608e7abf32a5708a8ed210da75893502ca84a529b1fe54680e1fcdbb801e06"
Jul 17 11:37:21.146325 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:21.146236    2566 generic.go:356] "Generic (PLEG): container finished" podID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerID="4202506c8c46892155e7be125f84df425b4ea14a37717ecb9ab6578847891793" exitCode=6
Jul 17 11:37:21.146796 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:21.146325    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" event={"ID":"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c","Type":"ContainerDied","Data":"4202506c8c46892155e7be125f84df425b4ea14a37717ecb9ab6578847891793"}
Jul 17 11:37:21.146796 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:21.146380    2566 scope.go:117] "RemoveContainer" containerID="06608e7abf32a5708a8ed210da75893502ca84a529b1fe54680e1fcdbb801e06"
Jul 17 11:37:21.146796 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:21.146546    2566 scope.go:117] "RemoveContainer" containerID="4202506c8c46892155e7be125f84df425b4ea14a37717ecb9ab6578847891793"
Jul 17 11:37:21.146796 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:21.146773    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl_odh-ai-gateway-infra(28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c"
Jul 17 11:37:21.148347 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:21.148331    2566 scope.go:117] "RemoveContainer" containerID="1b32269e501605d2915b30b36a0f1933214414f5cd7ae8d24fe2b88d7d6c4602"
Jul 17 11:37:21.148572 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:21.148552    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-pmrqr_odh-ai-gateway-infra(368cb778-dfc0-42fa-9c22-635c7b08dac6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6"
Jul 17 11:37:22.154209 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:22.154181    2566 scope.go:117] "RemoveContainer" containerID="4202506c8c46892155e7be125f84df425b4ea14a37717ecb9ab6578847891793"
Jul 17 11:37:22.154600 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:22.154387    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl_odh-ai-gateway-infra(28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c"
Jul 17 11:37:25.202796 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:25.202762    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"]
Jul 17 11:37:25.203213 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:25.202971    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy" containerID="cri-o://125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c" gracePeriod=30
Jul 17 11:37:27.968322 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:27.968291    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:37:30.456995 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.456969    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:37:30.617475 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617446    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-envoy\") pod \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") "
Jul 17 11:37:30.617637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617507    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-socket\") pod \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") "
Jul 17 11:37:30.617637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617544    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-podinfo\") pod \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") "
Jul 17 11:37:30.617637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617567    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-token\") pod \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") "
Jul 17 11:37:30.617637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617585    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-credential-socket\") pod \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") "
Jul 17 11:37:30.617836 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617643    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istiod-ca-cert\") pod \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") "
Jul 17 11:37:30.617836 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617666    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-data\") pod \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") "
Jul 17 11:37:30.617836 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617725    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4lp5\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-kube-api-access-b4lp5\") pod \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") "
Jul 17 11:37:30.617836 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617752    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-certs\") pod \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\" (UID: \"71a9c4ee-27fa-4649-a9c9-209a4ffed853\") "
Jul 17 11:37:30.618050 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617876    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71a9c4ee-27fa-4649-a9c9-209a4ffed853" (UID: "71a9c4ee-27fa-4649-a9c9-209a4ffed853"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:37:30.618050 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.617994    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71a9c4ee-27fa-4649-a9c9-209a4ffed853" (UID: "71a9c4ee-27fa-4649-a9c9-209a4ffed853"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:37:30.618050 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.618032    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71a9c4ee-27fa-4649-a9c9-209a4ffed853" (UID: "71a9c4ee-27fa-4649-a9c9-209a4ffed853"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:37:30.618218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.618121    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:30.618218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.618139    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:30.618218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.618149    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:30.618546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.618515    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71a9c4ee-27fa-4649-a9c9-209a4ffed853" (UID: "71a9c4ee-27fa-4649-a9c9-209a4ffed853"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:37:30.618717 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.618683    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-data" (OuterVolumeSpecName: "istio-data") pod "71a9c4ee-27fa-4649-a9c9-209a4ffed853" (UID: "71a9c4ee-27fa-4649-a9c9-209a4ffed853"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:37:30.619690 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.619670    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71a9c4ee-27fa-4649-a9c9-209a4ffed853" (UID: "71a9c4ee-27fa-4649-a9c9-209a4ffed853"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:37:30.619870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.619853    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71a9c4ee-27fa-4649-a9c9-209a4ffed853" (UID: "71a9c4ee-27fa-4649-a9c9-209a4ffed853"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:37:30.620115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.620093    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-token" (OuterVolumeSpecName: "istio-token") pod "71a9c4ee-27fa-4649-a9c9-209a4ffed853" (UID: "71a9c4ee-27fa-4649-a9c9-209a4ffed853"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:37:30.620200 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.620132    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-kube-api-access-b4lp5" (OuterVolumeSpecName: "kube-api-access-b4lp5") pod "71a9c4ee-27fa-4649-a9c9-209a4ffed853" (UID: "71a9c4ee-27fa-4649-a9c9-209a4ffed853"). InnerVolumeSpecName "kube-api-access-b4lp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:37:30.719013 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.718979    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4lp5\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-kube-api-access-b4lp5\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:30.719013 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.719010    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:30.719178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.719023    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:30.719178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.719035    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:30.719178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.719046    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:30.719178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:30.719056    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a9c4ee-27fa-4649-a9c9-209a4ffed853-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:31.197852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.197819    2566 generic.go:356] "Generic (PLEG): container finished" podID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerID="125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c" exitCode=0
Jul 17 11:37:31.198000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.197861    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"
Jul 17 11:37:31.198000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.197904    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" event={"ID":"71a9c4ee-27fa-4649-a9c9-209a4ffed853","Type":"ContainerDied","Data":"125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c"}
Jul 17 11:37:31.198000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.197955    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc" event={"ID":"71a9c4ee-27fa-4649-a9c9-209a4ffed853","Type":"ContainerDied","Data":"937e5199d0585258d0a0ec9054436f1f0cf5281befda042f4454f1ec6b7a8e0d"}
Jul 17 11:37:31.198000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.197978    2566 scope.go:117] "RemoveContainer" containerID="125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c"
Jul 17 11:37:31.207472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.207456    2566 scope.go:117] "RemoveContainer" containerID="7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51"
Jul 17 11:37:31.215089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.215074    2566 scope.go:117] "RemoveContainer" containerID="125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c"
Jul 17 11:37:31.215316 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:31.215299    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c\": container with ID starting with 125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c not found: ID does not exist" containerID="125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c"
Jul 17 11:37:31.215389 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.215323    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c"} err="failed to get container status \"125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c\": rpc error: code = NotFound desc = could not find container \"125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c\": container with ID starting with 125e74a443a6156be8ca2b9be9ef25e355e4a50ccdf696c36793323173cf6e8c not found: ID does not exist"
Jul 17 11:37:31.215389 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.215338    2566 scope.go:117] "RemoveContainer" containerID="7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51"
Jul 17 11:37:31.215563 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:31.215548    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51\": container with ID starting with 7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51 not found: ID does not exist" containerID="7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51"
Jul 17 11:37:31.215603 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.215567    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51"} err="failed to get container status \"7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51\": rpc error: code = NotFound desc = could not find container \"7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51\": container with ID starting with 7fd2efc723c7340c6056ccdae19e667d423a00ce289be1027c5f5c9ebae52e51 not found: ID does not exist"
Jul 17 11:37:31.245700 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.245673    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"]
Jul 17 11:37:31.247283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.247264    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-57741a73-openshift-default-676cd77759-vd8rc"]
Jul 17 11:37:31.657617 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:31.657583    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" path="/var/lib/kubelet/pods/71a9c4ee-27fa-4649-a9c9-209a4ffed853/volumes"
Jul 17 11:37:34.652422 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:34.652393    2566 scope.go:117] "RemoveContainer" containerID="1b32269e501605d2915b30b36a0f1933214414f5cd7ae8d24fe2b88d7d6c4602"
Jul 17 11:37:35.220229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:35.220196    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" event={"ID":"368cb778-dfc0-42fa-9c22-635c7b08dac6","Type":"ContainerStarted","Data":"06e599b1742bcc50e2dc56744dc6f4aa82cc51f65d691bd23913a9290ec6a401"}
Jul 17 11:37:36.229950 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:36.229915    2566 generic.go:356] "Generic (PLEG): container finished" podID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerID="06e599b1742bcc50e2dc56744dc6f4aa82cc51f65d691bd23913a9290ec6a401" exitCode=6
Jul 17 11:37:36.230360 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:36.229985    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" event={"ID":"368cb778-dfc0-42fa-9c22-635c7b08dac6","Type":"ContainerDied","Data":"06e599b1742bcc50e2dc56744dc6f4aa82cc51f65d691bd23913a9290ec6a401"}
Jul 17 11:37:36.230360 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:36.230058    2566 scope.go:117] "RemoveContainer" containerID="1b32269e501605d2915b30b36a0f1933214414f5cd7ae8d24fe2b88d7d6c4602"
Jul 17 11:37:36.230360 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:36.230354    2566 scope.go:117] "RemoveContainer" containerID="06e599b1742bcc50e2dc56744dc6f4aa82cc51f65d691bd23913a9290ec6a401"
Jul 17 11:37:36.230591 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:36.230571    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-pmrqr_odh-ai-gateway-infra(368cb778-dfc0-42fa-9c22-635c7b08dac6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6"
Jul 17 11:37:36.288096 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:36.288071    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"]
Jul 17 11:37:36.652124 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:36.652090    2566 scope.go:117] "RemoveContainer" containerID="4202506c8c46892155e7be125f84df425b4ea14a37717ecb9ab6578847891793"
Jul 17 11:37:37.239100 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:37.239074    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" event={"ID":"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c","Type":"ContainerStarted","Data":"ff7d091130302a90f4e269be9dd46ec44940f259899de2bc6bfa7d2adf277f13"}
Jul 17 11:37:37.326731 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:37.324974    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" podStartSLOduration=19.324953563 podStartE2EDuration="19.324953563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:37:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:37:37.321010214 +0000 UTC m=+2440.178094159" watchObservedRunningTime="2026-07-17 11:37:37.324953563 +0000 UTC m=+2440.182037508"
Jul 17 11:37:37.367214 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:37.367189    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"
Jul 17 11:37:37.481174 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:37.481148    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368cb778-dfc0-42fa-9c22-635c7b08dac6-maas-api-service-ca\") pod \"368cb778-dfc0-42fa-9c22-635c7b08dac6\" (UID: \"368cb778-dfc0-42fa-9c22-635c7b08dac6\") "
Jul 17 11:37:37.481477 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:37.481456    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/368cb778-dfc0-42fa-9c22-635c7b08dac6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "368cb778-dfc0-42fa-9c22-635c7b08dac6" (UID: "368cb778-dfc0-42fa-9c22-635c7b08dac6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:37:37.582570 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:37.582546    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368cb778-dfc0-42fa-9c22-635c7b08dac6-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:38.244909 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.244877    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"
Jul 17 11:37:38.244909 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.244893    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr" event={"ID":"368cb778-dfc0-42fa-9c22-635c7b08dac6","Type":"ContainerDied","Data":"f4ad16bd9688c458e952f1dd3579cca6565c4edf819ccc3215c8e9483f86b320"}
Jul 17 11:37:38.245444 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.244952    2566 scope.go:117] "RemoveContainer" containerID="06e599b1742bcc50e2dc56744dc6f4aa82cc51f65d691bd23913a9290ec6a401"
Jul 17 11:37:38.247085 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.247044    2566 generic.go:356] "Generic (PLEG): container finished" podID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerID="ff7d091130302a90f4e269be9dd46ec44940f259899de2bc6bfa7d2adf277f13" exitCode=6
Jul 17 11:37:38.247225 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.247136    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" event={"ID":"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c","Type":"ContainerDied","Data":"ff7d091130302a90f4e269be9dd46ec44940f259899de2bc6bfa7d2adf277f13"}
Jul 17 11:37:38.247466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.247450    2566 scope.go:117] "RemoveContainer" containerID="ff7d091130302a90f4e269be9dd46ec44940f259899de2bc6bfa7d2adf277f13"
Jul 17 11:37:38.247646 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:38.247628    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl_odh-ai-gateway-infra(28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c"
Jul 17 11:37:38.253356 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.253340    2566 scope.go:117] "RemoveContainer" containerID="4202506c8c46892155e7be125f84df425b4ea14a37717ecb9ab6578847891793"
Jul 17 11:37:38.291504 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.291479    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"]
Jul 17 11:37:38.311825 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.311804    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-pmrqr"]
Jul 17 11:37:38.346554 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.346530    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"]
Jul 17 11:37:38.418180 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418158    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"]
Jul 17 11:37:38.418495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418484    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerName="revoke-keys"
Jul 17 11:37:38.418548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418497    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerName="revoke-keys"
Jul 17 11:37:38.418548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418510    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy"
Jul 17 11:37:38.418548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418516    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy"
Jul 17 11:37:38.418548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418524    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy"
Jul 17 11:37:38.418548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418529    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy"
Jul 17 11:37:38.418548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418537    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerName="revoke-keys"
Jul 17 11:37:38.418548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418542    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerName="revoke-keys"
Jul 17 11:37:38.418821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418606    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy"
Jul 17 11:37:38.418821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418620    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerName="revoke-keys"
Jul 17 11:37:38.418821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418627    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerName="revoke-keys"
Jul 17 11:37:38.418821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.418633    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerName="revoke-keys"
Jul 17 11:37:38.422571 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.422554    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"
Jul 17 11:37:38.477071 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.477050    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"]
Jul 17 11:37:38.591439 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.591416    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/593642e6-7219-4c71-8843-09a543e4ef58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv\" (UID: \"593642e6-7219-4c71-8843-09a543e4ef58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"
Jul 17 11:37:38.692790 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.692762    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/593642e6-7219-4c71-8843-09a543e4ef58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv\" (UID: \"593642e6-7219-4c71-8843-09a543e4ef58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"
Jul 17 11:37:38.693399 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.693373    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/593642e6-7219-4c71-8843-09a543e4ef58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv\" (UID: \"593642e6-7219-4c71-8843-09a543e4ef58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"
Jul 17 11:37:38.732723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:38.732676    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"
Jul 17 11:37:39.092810 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:39.092779    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"]
Jul 17 11:37:39.095599 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:37:39.095571    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593642e6_7219_4c71_8843_09a543e4ef58.slice/crio-7a5e0d3ded4d32ea0a3e13a52b1551970fcd5cbb05689483ba479d0992bfeeb7 WatchSource:0}: Error finding container 7a5e0d3ded4d32ea0a3e13a52b1551970fcd5cbb05689483ba479d0992bfeeb7: Status 404 returned error can't find the container with id 7a5e0d3ded4d32ea0a3e13a52b1551970fcd5cbb05689483ba479d0992bfeeb7
Jul 17 11:37:39.254899 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:39.254868    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" event={"ID":"593642e6-7219-4c71-8843-09a543e4ef58","Type":"ContainerStarted","Data":"7a5e0d3ded4d32ea0a3e13a52b1551970fcd5cbb05689483ba479d0992bfeeb7"}
Jul 17 11:37:39.385247 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:39.385223    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"
Jul 17 11:37:39.500299 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:39.500276    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c-maas-api-service-ca\") pod \"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c\" (UID: \"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c\") "
Jul 17 11:37:39.500616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:39.500592    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" (UID: "28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:37:39.601622 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:39.601599    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:37:39.656983 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:39.656925    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" path="/var/lib/kubelet/pods/368cb778-dfc0-42fa-9c22-635c7b08dac6/volumes"
Jul 17 11:37:40.261718 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:40.261665    2566 generic.go:356] "Generic (PLEG): container finished" podID="593642e6-7219-4c71-8843-09a543e4ef58" containerID="cc0a48fc8c4797548ae61de78da44d5534897cf0d2a8ec51ef9941163ba64a86" exitCode=6
Jul 17 11:37:40.262193 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:40.261739    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" event={"ID":"593642e6-7219-4c71-8843-09a543e4ef58","Type":"ContainerDied","Data":"cc0a48fc8c4797548ae61de78da44d5534897cf0d2a8ec51ef9941163ba64a86"}
Jul 17 11:37:40.262193 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:40.261947    2566 scope.go:117] "RemoveContainer" containerID="cc0a48fc8c4797548ae61de78da44d5534897cf0d2a8ec51ef9941163ba64a86"
Jul 17 11:37:40.263061 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:40.263038    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"
Jul 17 11:37:40.263181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:40.263064    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl" event={"ID":"28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c","Type":"ContainerDied","Data":"c009c51b24ded6c70542bc5098f889f7190ee1a82cb9912acccba8e5efab1f65"}
Jul 17 11:37:40.263181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:40.263114    2566 scope.go:117] "RemoveContainer" containerID="ff7d091130302a90f4e269be9dd46ec44940f259899de2bc6bfa7d2adf277f13"
Jul 17 11:37:40.325883 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:40.325856    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"]
Jul 17 11:37:40.330905 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:40.330884    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-7v9jl"]
Jul 17 11:37:41.269965 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:41.269874    2566 generic.go:356] "Generic (PLEG): container finished" podID="593642e6-7219-4c71-8843-09a543e4ef58" containerID="0fd81435a183767432e30272e64ba470269dfed439e50ca0cc3f78defe5e54a2" exitCode=6
Jul 17 11:37:41.270412 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:41.269958    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" event={"ID":"593642e6-7219-4c71-8843-09a543e4ef58","Type":"ContainerDied","Data":"0fd81435a183767432e30272e64ba470269dfed439e50ca0cc3f78defe5e54a2"}
Jul 17 11:37:41.270412 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:41.270017    2566 scope.go:117] "RemoveContainer" containerID="cc0a48fc8c4797548ae61de78da44d5534897cf0d2a8ec51ef9941163ba64a86"
Jul 17 11:37:41.270412 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:41.270171    2566 scope.go:117] "RemoveContainer" containerID="0fd81435a183767432e30272e64ba470269dfed439e50ca0cc3f78defe5e54a2"
Jul 17 11:37:41.270412 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:41.270385    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv_odh-ai-gateway-infra(593642e6-7219-4c71-8843-09a543e4ef58)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" podUID="593642e6-7219-4c71-8843-09a543e4ef58"
Jul 17 11:37:41.658043 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:41.658007    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" path="/var/lib/kubelet/pods/28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c/volumes"
Jul 17 11:37:42.278559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:42.278528    2566 scope.go:117] "RemoveContainer" containerID="0fd81435a183767432e30272e64ba470269dfed439e50ca0cc3f78defe5e54a2"
Jul 17 11:37:42.279061 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:42.278803    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv_odh-ai-gateway-infra(593642e6-7219-4c71-8843-09a543e4ef58)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" podUID="593642e6-7219-4c71-8843-09a543e4ef58"
Jul 17 11:37:55.227992 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.227958    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"]
Jul 17 11:37:55.228369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228330    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerName="revoke-keys"
Jul 17 11:37:55.228369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228340    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="368cb778-dfc0-42fa-9c22-635c7b08dac6" containerName="revoke-keys"
Jul 17 11:37:55.228369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228365    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerName="revoke-keys"
Jul 17 11:37:55.228369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228371    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerName="revoke-keys"
Jul 17 11:37:55.228528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228379    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerName="revoke-keys"
Jul 17 11:37:55.228528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228385    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerName="revoke-keys"
Jul 17 11:37:55.228528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228394    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerName="revoke-keys"
Jul 17 11:37:55.228528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228400    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerName="revoke-keys"
Jul 17 11:37:55.228528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228453    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="71a9c4ee-27fa-4649-a9c9-209a4ffed853" containerName="istio-proxy"
Jul 17 11:37:55.228528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228461    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerName="revoke-keys"
Jul 17 11:37:55.228528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228467    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerName="revoke-keys"
Jul 17 11:37:55.228528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.228474    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ec4d65-ffd4-4a4b-a9b7-0f49130e6b0c" containerName="revoke-keys"
Jul 17 11:37:55.232826 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.232804    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"
Jul 17 11:37:55.268963 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.268903    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"]
Jul 17 11:37:55.319111 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.319079    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a399911f-ce20-404a-9329-1b8dd7335464-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-jz6gh\" (UID: \"a399911f-ce20-404a-9329-1b8dd7335464\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"
Jul 17 11:37:55.420065 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.420036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a399911f-ce20-404a-9329-1b8dd7335464-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-jz6gh\" (UID: \"a399911f-ce20-404a-9329-1b8dd7335464\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"
Jul 17 11:37:55.420632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.420612    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a399911f-ce20-404a-9329-1b8dd7335464-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-jz6gh\" (UID: \"a399911f-ce20-404a-9329-1b8dd7335464\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"
Jul 17 11:37:55.543812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.543742    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"
Jul 17 11:37:55.895745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:55.895688    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"]
Jul 17 11:37:55.898526 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:37:55.898491    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda399911f_ce20_404a_9329_1b8dd7335464.slice/crio-932c988875583d79c751cc9cf36124b0359881a75560bbbbbca624e93c3266d7 WatchSource:0}: Error finding container 932c988875583d79c751cc9cf36124b0359881a75560bbbbbca624e93c3266d7: Status 404 returned error can't find the container with id 932c988875583d79c751cc9cf36124b0359881a75560bbbbbca624e93c3266d7
Jul 17 11:37:56.348181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:56.348150    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" event={"ID":"a399911f-ce20-404a-9329-1b8dd7335464","Type":"ContainerStarted","Data":"7290b8f59cacb4b40e872fdc4dee57c136fb1f86ffa0ded905fb415ad5629df5"}
Jul 17 11:37:56.348566 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:56.348200    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" event={"ID":"a399911f-ce20-404a-9329-1b8dd7335464","Type":"ContainerStarted","Data":"932c988875583d79c751cc9cf36124b0359881a75560bbbbbca624e93c3266d7"}
Jul 17 11:37:56.390227 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:56.390182    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" podStartSLOduration=1.3901650509999999 podStartE2EDuration="1.390165051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:37:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:37:56.388458119 +0000 UTC m=+2459.245542076" watchObservedRunningTime="2026-07-17 11:37:56.390165051 +0000 UTC m=+2459.247248973"
Jul 17 11:37:57.354757 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:57.354698    2566 generic.go:356] "Generic (PLEG): container finished" podID="a399911f-ce20-404a-9329-1b8dd7335464" containerID="7290b8f59cacb4b40e872fdc4dee57c136fb1f86ffa0ded905fb415ad5629df5" exitCode=6
Jul 17 11:37:57.355186 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:57.354800    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" event={"ID":"a399911f-ce20-404a-9329-1b8dd7335464","Type":"ContainerDied","Data":"7290b8f59cacb4b40e872fdc4dee57c136fb1f86ffa0ded905fb415ad5629df5"}
Jul 17 11:37:57.355186 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:57.354969    2566 scope.go:117] "RemoveContainer" containerID="7290b8f59cacb4b40e872fdc4dee57c136fb1f86ffa0ded905fb415ad5629df5"
Jul 17 11:37:57.654671 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:57.654593    2566 scope.go:117] "RemoveContainer" containerID="0fd81435a183767432e30272e64ba470269dfed439e50ca0cc3f78defe5e54a2"
Jul 17 11:37:58.360794 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:58.360762    2566 generic.go:356] "Generic (PLEG): container finished" podID="a399911f-ce20-404a-9329-1b8dd7335464" containerID="c3efce896d2cc99c1d8b9f8ac04d66b6ed33631fce7e01bad87d02c090faf3e0" exitCode=6
Jul 17 11:37:58.361237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:58.360845    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" event={"ID":"a399911f-ce20-404a-9329-1b8dd7335464","Type":"ContainerDied","Data":"c3efce896d2cc99c1d8b9f8ac04d66b6ed33631fce7e01bad87d02c090faf3e0"}
Jul 17 11:37:58.361237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:58.360911    2566 scope.go:117] "RemoveContainer" containerID="7290b8f59cacb4b40e872fdc4dee57c136fb1f86ffa0ded905fb415ad5629df5"
Jul 17 11:37:58.361237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:58.361035    2566 scope.go:117] "RemoveContainer" containerID="c3efce896d2cc99c1d8b9f8ac04d66b6ed33631fce7e01bad87d02c090faf3e0"
Jul 17 11:37:58.361404 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:58.361256    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-jz6gh_odh-ai-gateway-infra(a399911f-ce20-404a-9329-1b8dd7335464)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" podUID="a399911f-ce20-404a-9329-1b8dd7335464"
Jul 17 11:37:58.362621 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:58.362599    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" event={"ID":"593642e6-7219-4c71-8843-09a543e4ef58","Type":"ContainerStarted","Data":"cf82d01dbd5aef1e5511f7ab491c649505b01f11c2d95675ef1952da9ad2775e"}
Jul 17 11:37:58.481195 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:58.481155    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" podStartSLOduration=20.481141236 podStartE2EDuration="20.481141236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:37:58.478839921 +0000 UTC m=+2461.335923865" watchObservedRunningTime="2026-07-17 11:37:58.481141236 +0000 UTC m=+2461.338225181"
Jul 17 11:37:59.368823 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:59.368788    2566 generic.go:356] "Generic (PLEG): container finished" podID="593642e6-7219-4c71-8843-09a543e4ef58" containerID="cf82d01dbd5aef1e5511f7ab491c649505b01f11c2d95675ef1952da9ad2775e" exitCode=6
Jul 17 11:37:59.369244 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:59.368856    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" event={"ID":"593642e6-7219-4c71-8843-09a543e4ef58","Type":"ContainerDied","Data":"cf82d01dbd5aef1e5511f7ab491c649505b01f11c2d95675ef1952da9ad2775e"}
Jul 17 11:37:59.369244 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:59.368921    2566 scope.go:117] "RemoveContainer" containerID="0fd81435a183767432e30272e64ba470269dfed439e50ca0cc3f78defe5e54a2"
Jul 17 11:37:59.369244 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:59.369198    2566 scope.go:117] "RemoveContainer" containerID="cf82d01dbd5aef1e5511f7ab491c649505b01f11c2d95675ef1952da9ad2775e"
Jul 17 11:37:59.369435 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:59.369416    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv_odh-ai-gateway-infra(593642e6-7219-4c71-8843-09a543e4ef58)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" podUID="593642e6-7219-4c71-8843-09a543e4ef58"
Jul 17 11:37:59.370681 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:59.370665    2566 scope.go:117] "RemoveContainer" containerID="c3efce896d2cc99c1d8b9f8ac04d66b6ed33631fce7e01bad87d02c090faf3e0"
Jul 17 11:37:59.370850 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:37:59.370833    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-jz6gh_odh-ai-gateway-infra(a399911f-ce20-404a-9329-1b8dd7335464)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" podUID="a399911f-ce20-404a-9329-1b8dd7335464"
Jul 17 11:37:59.500516 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:37:59.500493    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"]
Jul 17 11:38:00.509301 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:00.509277    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"
Jul 17 11:38:00.558817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:00.558786    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/593642e6-7219-4c71-8843-09a543e4ef58-maas-api-service-ca\") pod \"593642e6-7219-4c71-8843-09a543e4ef58\" (UID: \"593642e6-7219-4c71-8843-09a543e4ef58\") "
Jul 17 11:38:00.559127 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:00.559105    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/593642e6-7219-4c71-8843-09a543e4ef58-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "593642e6-7219-4c71-8843-09a543e4ef58" (UID: "593642e6-7219-4c71-8843-09a543e4ef58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:38:00.660112 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:00.660020    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/593642e6-7219-4c71-8843-09a543e4ef58-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:38:01.383155 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:01.383128    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"
Jul 17 11:38:01.383443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:01.383129    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv" event={"ID":"593642e6-7219-4c71-8843-09a543e4ef58","Type":"ContainerDied","Data":"7a5e0d3ded4d32ea0a3e13a52b1551970fcd5cbb05689483ba479d0992bfeeb7"}
Jul 17 11:38:01.383443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:01.383274    2566 scope.go:117] "RemoveContainer" containerID="cf82d01dbd5aef1e5511f7ab491c649505b01f11c2d95675ef1952da9ad2775e"
Jul 17 11:38:01.418758 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:01.418729    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"]
Jul 17 11:38:01.428187 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:01.428163    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-dcxkv"]
Jul 17 11:38:01.659325 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:01.659244    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="593642e6-7219-4c71-8843-09a543e4ef58" path="/var/lib/kubelet/pods/593642e6-7219-4c71-8843-09a543e4ef58/volumes"
Jul 17 11:38:13.652598 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:13.652564    2566 scope.go:117] "RemoveContainer" containerID="c3efce896d2cc99c1d8b9f8ac04d66b6ed33631fce7e01bad87d02c090faf3e0"
Jul 17 11:38:14.447038 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:14.447002    2566 generic.go:356] "Generic (PLEG): container finished" podID="a399911f-ce20-404a-9329-1b8dd7335464" containerID="c4739ec630ab66dc416da266b38d2817d3770338dace219a6fc5e404ce505a0d" exitCode=6
Jul 17 11:38:14.447200 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:14.447072    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" event={"ID":"a399911f-ce20-404a-9329-1b8dd7335464","Type":"ContainerDied","Data":"c4739ec630ab66dc416da266b38d2817d3770338dace219a6fc5e404ce505a0d"}
Jul 17 11:38:14.447200 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:14.447157    2566 scope.go:117] "RemoveContainer" containerID="c3efce896d2cc99c1d8b9f8ac04d66b6ed33631fce7e01bad87d02c090faf3e0"
Jul 17 11:38:14.447453 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:14.447435    2566 scope.go:117] "RemoveContainer" containerID="c4739ec630ab66dc416da266b38d2817d3770338dace219a6fc5e404ce505a0d"
Jul 17 11:38:14.447659 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:14.447632    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-jz6gh_odh-ai-gateway-infra(a399911f-ce20-404a-9329-1b8dd7335464)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" podUID="a399911f-ce20-404a-9329-1b8dd7335464"
Jul 17 11:38:15.509870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:15.509841    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"]
Jul 17 11:38:15.649944 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:15.649923    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"
Jul 17 11:38:15.791499 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:15.791418    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a399911f-ce20-404a-9329-1b8dd7335464-maas-api-service-ca\") pod \"a399911f-ce20-404a-9329-1b8dd7335464\" (UID: \"a399911f-ce20-404a-9329-1b8dd7335464\") "
Jul 17 11:38:15.791775 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:15.791753    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a399911f-ce20-404a-9329-1b8dd7335464-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a399911f-ce20-404a-9329-1b8dd7335464" (UID: "a399911f-ce20-404a-9329-1b8dd7335464"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:38:15.892719 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:15.892671    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a399911f-ce20-404a-9329-1b8dd7335464-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:38:16.459017 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:16.458984    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"
Jul 17 11:38:16.459190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:16.459028    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh" event={"ID":"a399911f-ce20-404a-9329-1b8dd7335464","Type":"ContainerDied","Data":"932c988875583d79c751cc9cf36124b0359881a75560bbbbbca624e93c3266d7"}
Jul 17 11:38:16.459190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:16.459087    2566 scope.go:117] "RemoveContainer" containerID="c4739ec630ab66dc416da266b38d2817d3770338dace219a6fc5e404ce505a0d"
Jul 17 11:38:16.493781 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:16.493758    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"]
Jul 17 11:38:16.499370 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:16.499348    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-jz6gh"]
Jul 17 11:38:17.323156 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323121    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"]
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323469    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="593642e6-7219-4c71-8843-09a543e4ef58" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323479    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="593642e6-7219-4c71-8843-09a543e4ef58" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323487    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a399911f-ce20-404a-9329-1b8dd7335464" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323493    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a399911f-ce20-404a-9329-1b8dd7335464" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323501    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a399911f-ce20-404a-9329-1b8dd7335464" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323508    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a399911f-ce20-404a-9329-1b8dd7335464" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323530    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="593642e6-7219-4c71-8843-09a543e4ef58" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323535    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="593642e6-7219-4c71-8843-09a543e4ef58" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323588    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="593642e6-7219-4c71-8843-09a543e4ef58" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323594    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a399911f-ce20-404a-9329-1b8dd7335464" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323601    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="593642e6-7219-4c71-8843-09a543e4ef58" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323608    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="593642e6-7219-4c71-8843-09a543e4ef58" containerName="revoke-keys"
Jul 17 11:38:17.323611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323614    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a399911f-ce20-404a-9329-1b8dd7335464" containerName="revoke-keys"
Jul 17 11:38:17.324223 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.323620    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a399911f-ce20-404a-9329-1b8dd7335464" containerName="revoke-keys"
Jul 17 11:38:17.328018 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.328002    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"
Jul 17 11:38:17.331415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.331389    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:38:17.331655 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.331638    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:38:17.344418 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.344390    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"]
Jul 17 11:38:17.404093 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.404056    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-6m2bx\" (UID: \"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"
Jul 17 11:38:17.505239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.505208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-6m2bx\" (UID: \"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"
Jul 17 11:38:17.505867 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.505847    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-6m2bx\" (UID: \"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"
Jul 17 11:38:17.638783 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.638666    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"
Jul 17 11:38:17.659474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.659443    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a399911f-ce20-404a-9329-1b8dd7335464" path="/var/lib/kubelet/pods/a399911f-ce20-404a-9329-1b8dd7335464/volumes"
Jul 17 11:38:17.980066 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:17.979990    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"]
Jul 17 11:38:17.982767 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:38:17.982739    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8fcc1c_ba76_4333_8dab_7b6fa61f11c1.slice/crio-8fa161581bd1e2073bfb982b9fc5ef003db36d31412d2a9596906bbecb39ad9e WatchSource:0}: Error finding container 8fa161581bd1e2073bfb982b9fc5ef003db36d31412d2a9596906bbecb39ad9e: Status 404 returned error can't find the container with id 8fa161581bd1e2073bfb982b9fc5ef003db36d31412d2a9596906bbecb39ad9e
Jul 17 11:38:18.472471 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.472424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" event={"ID":"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1","Type":"ContainerStarted","Data":"5e222c943fc25e7f264c30aba5be81fb574ade1d2ae5a89af1027fd3620014ef"}
Jul 17 11:38:18.472907 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.472513    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" event={"ID":"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1","Type":"ContainerStarted","Data":"8fa161581bd1e2073bfb982b9fc5ef003db36d31412d2a9596906bbecb39ad9e"}
Jul 17 11:38:18.503064 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.503015    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" podStartSLOduration=1.503000939 podStartE2EDuration="1.503000939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:38:18.500051766 +0000 UTC m=+2481.357135705" watchObservedRunningTime="2026-07-17 11:38:18.503000939 +0000 UTC m=+2481.360084883"
Jul 17 11:38:18.531450 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.531426    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"]
Jul 17 11:38:18.531800 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.531787    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="593642e6-7219-4c71-8843-09a543e4ef58" containerName="revoke-keys"
Jul 17 11:38:18.531800 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.531801    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="593642e6-7219-4c71-8843-09a543e4ef58" containerName="revoke-keys"
Jul 17 11:38:18.531898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.531813    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a399911f-ce20-404a-9329-1b8dd7335464" containerName="revoke-keys"
Jul 17 11:38:18.531898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.531819    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a399911f-ce20-404a-9329-1b8dd7335464" containerName="revoke-keys"
Jul 17 11:38:18.535026 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.534917    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"
Jul 17 11:38:18.551987 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.551952    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"]
Jul 17 11:38:18.616847 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.616819    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f784e7ff-bf48-4937-8966-ffe0571c93cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk\" (UID: \"f784e7ff-bf48-4937-8966-ffe0571c93cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"
Jul 17 11:38:18.718060 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.718024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f784e7ff-bf48-4937-8966-ffe0571c93cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk\" (UID: \"f784e7ff-bf48-4937-8966-ffe0571c93cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"
Jul 17 11:38:18.718639 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.718619    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f784e7ff-bf48-4937-8966-ffe0571c93cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk\" (UID: \"f784e7ff-bf48-4937-8966-ffe0571c93cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"
Jul 17 11:38:18.896213 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:18.896179    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"
Jul 17 11:38:19.044468 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:19.044443    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"]
Jul 17 11:38:19.046464 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:38:19.046437    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf784e7ff_bf48_4937_8966_ffe0571c93cd.slice/crio-7090480f24b05bf2e0521e4d4ce7e40d96f6baead02966e51c610a3fe0716360 WatchSource:0}: Error finding container 7090480f24b05bf2e0521e4d4ce7e40d96f6baead02966e51c610a3fe0716360: Status 404 returned error can't find the container with id 7090480f24b05bf2e0521e4d4ce7e40d96f6baead02966e51c610a3fe0716360
Jul 17 11:38:19.478970 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:19.478937    2566 generic.go:356] "Generic (PLEG): container finished" podID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerID="5e222c943fc25e7f264c30aba5be81fb574ade1d2ae5a89af1027fd3620014ef" exitCode=6
Jul 17 11:38:19.479411 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:19.479031    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" event={"ID":"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1","Type":"ContainerDied","Data":"5e222c943fc25e7f264c30aba5be81fb574ade1d2ae5a89af1027fd3620014ef"}
Jul 17 11:38:19.479411 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:19.479205    2566 scope.go:117] "RemoveContainer" containerID="5e222c943fc25e7f264c30aba5be81fb574ade1d2ae5a89af1027fd3620014ef"
Jul 17 11:38:19.480587 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:19.480566    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk" event={"ID":"f784e7ff-bf48-4937-8966-ffe0571c93cd","Type":"ContainerStarted","Data":"ab3159b6d130a28df7741dcfd2ea27b85fcb3b7063becf8eec436b0bc4ad4245"}
Jul 17 11:38:19.480739 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:19.480606    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk" event={"ID":"f784e7ff-bf48-4937-8966-ffe0571c93cd","Type":"ContainerStarted","Data":"7090480f24b05bf2e0521e4d4ce7e40d96f6baead02966e51c610a3fe0716360"}
Jul 17 11:38:20.487066 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:20.487032    2566 generic.go:356] "Generic (PLEG): container finished" podID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerID="ab3159b6d130a28df7741dcfd2ea27b85fcb3b7063becf8eec436b0bc4ad4245" exitCode=6
Jul 17 11:38:20.487528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:20.487116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk" event={"ID":"f784e7ff-bf48-4937-8966-ffe0571c93cd","Type":"ContainerDied","Data":"ab3159b6d130a28df7741dcfd2ea27b85fcb3b7063becf8eec436b0bc4ad4245"}
Jul 17 11:38:20.487528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:20.487331    2566 scope.go:117] "RemoveContainer" containerID="ab3159b6d130a28df7741dcfd2ea27b85fcb3b7063becf8eec436b0bc4ad4245"
Jul 17 11:38:20.488876 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:20.488855    2566 generic.go:356] "Generic (PLEG): container finished" podID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerID="e7ef5af2e068f3f2187f32fe4cebdc525a84f7f332674496f82c373e3ffed6a7" exitCode=6
Jul 17 11:38:20.488993 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:20.488880    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" event={"ID":"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1","Type":"ContainerDied","Data":"e7ef5af2e068f3f2187f32fe4cebdc525a84f7f332674496f82c373e3ffed6a7"}
Jul 17 11:38:20.488993 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:20.488912    2566 scope.go:117] "RemoveContainer" containerID="5e222c943fc25e7f264c30aba5be81fb574ade1d2ae5a89af1027fd3620014ef"
Jul 17 11:38:20.489085 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:20.489069    2566 scope.go:117] "RemoveContainer" containerID="e7ef5af2e068f3f2187f32fe4cebdc525a84f7f332674496f82c373e3ffed6a7"
Jul 17 11:38:20.489310 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:20.489286    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-6m2bx_odh-ai-gateway-infra(dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1"
Jul 17 11:38:21.496331 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:21.496246    2566 scope.go:117] "RemoveContainer" containerID="e7ef5af2e068f3f2187f32fe4cebdc525a84f7f332674496f82c373e3ffed6a7"
Jul 17 11:38:21.496760 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:21.496485    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-6m2bx_odh-ai-gateway-infra(dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1"
Jul 17 11:38:21.497676 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:21.497649    2566 generic.go:356] "Generic (PLEG): container finished" podID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerID="c74f82354e613aa10f1644034d5bbd7fc76460fd1c205baeefdfccde3fc4f34c" exitCode=6
Jul 17 11:38:21.497807 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:21.497685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk" event={"ID":"f784e7ff-bf48-4937-8966-ffe0571c93cd","Type":"ContainerDied","Data":"c74f82354e613aa10f1644034d5bbd7fc76460fd1c205baeefdfccde3fc4f34c"}
Jul 17 11:38:21.497807 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:21.497756    2566 scope.go:117] "RemoveContainer" containerID="ab3159b6d130a28df7741dcfd2ea27b85fcb3b7063becf8eec436b0bc4ad4245"
Jul 17 11:38:21.497923 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:21.497894    2566 scope.go:117] "RemoveContainer" containerID="c74f82354e613aa10f1644034d5bbd7fc76460fd1c205baeefdfccde3fc4f34c"
Jul 17 11:38:21.498086 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:21.498069    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk_odh-ai-gateway-infra(f784e7ff-bf48-4937-8966-ffe0571c93cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd"
Jul 17 11:38:22.504734 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:22.504695    2566 scope.go:117] "RemoveContainer" containerID="c74f82354e613aa10f1644034d5bbd7fc76460fd1c205baeefdfccde3fc4f34c"
Jul 17 11:38:22.505160 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:22.504970    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk_odh-ai-gateway-infra(f784e7ff-bf48-4937-8966-ffe0571c93cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd"
Jul 17 11:38:35.652229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:35.652194    2566 scope.go:117] "RemoveContainer" containerID="e7ef5af2e068f3f2187f32fe4cebdc525a84f7f332674496f82c373e3ffed6a7"
Jul 17 11:38:36.572189 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:36.572167    2566 generic.go:356] "Generic (PLEG): container finished" podID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerID="89c03123c3ebe5be2d7199198210d555457a317d9ce36a759eb0ff639e77b381" exitCode=6
Jul 17 11:38:36.572329 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:36.572245    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" event={"ID":"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1","Type":"ContainerDied","Data":"89c03123c3ebe5be2d7199198210d555457a317d9ce36a759eb0ff639e77b381"}
Jul 17 11:38:36.572329 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:36.572306    2566 scope.go:117] "RemoveContainer" containerID="e7ef5af2e068f3f2187f32fe4cebdc525a84f7f332674496f82c373e3ffed6a7"
Jul 17 11:38:36.572586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:36.572565    2566 scope.go:117] "RemoveContainer" containerID="89c03123c3ebe5be2d7199198210d555457a317d9ce36a759eb0ff639e77b381"
Jul 17 11:38:36.572825 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:36.572802    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-6m2bx_odh-ai-gateway-infra(dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1"
Jul 17 11:38:36.652269 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:36.652242    2566 scope.go:117] "RemoveContainer" containerID="c74f82354e613aa10f1644034d5bbd7fc76460fd1c205baeefdfccde3fc4f34c"
Jul 17 11:38:37.579553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:37.579525    2566 generic.go:356] "Generic (PLEG): container finished" podID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerID="7a03c80af615b8f13b49609eab3391005ae2f1e7ba17c5d8b42a73f4e963c13b" exitCode=6
Jul 17 11:38:37.579696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:37.579590    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk" event={"ID":"f784e7ff-bf48-4937-8966-ffe0571c93cd","Type":"ContainerDied","Data":"7a03c80af615b8f13b49609eab3391005ae2f1e7ba17c5d8b42a73f4e963c13b"}
Jul 17 11:38:37.579696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:37.579644    2566 scope.go:117] "RemoveContainer" containerID="c74f82354e613aa10f1644034d5bbd7fc76460fd1c205baeefdfccde3fc4f34c"
Jul 17 11:38:37.579955 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:37.579935    2566 scope.go:117] "RemoveContainer" containerID="7a03c80af615b8f13b49609eab3391005ae2f1e7ba17c5d8b42a73f4e963c13b"
Jul 17 11:38:37.580131 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:37.580114    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk_odh-ai-gateway-infra(f784e7ff-bf48-4937-8966-ffe0571c93cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd"
Jul 17 11:38:37.605119 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:37.605090    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"]
Jul 17 11:38:37.741063 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:37.741039    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"
Jul 17 11:38:37.888619 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:37.888531    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1-maas-api-service-ca\") pod \"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1\" (UID: \"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1\") "
Jul 17 11:38:37.888918 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:37.888895    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" (UID: "dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:38:37.989413 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:37.989380    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:38:38.470931 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.470898    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"]
Jul 17 11:38:38.471269 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.471257    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerName="revoke-keys"
Jul 17 11:38:38.471318 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.471269    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerName="revoke-keys"
Jul 17 11:38:38.471318 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.471278    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerName="revoke-keys"
Jul 17 11:38:38.471318 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.471286    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerName="revoke-keys"
Jul 17 11:38:38.471408 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.471353    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerName="revoke-keys"
Jul 17 11:38:38.471408 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.471359    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerName="revoke-keys"
Jul 17 11:38:38.474270 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.474251    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"
Jul 17 11:38:38.490001 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.489980    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"]
Jul 17 11:38:38.586085 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.586057    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"
Jul 17 11:38:38.586085 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.586079    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx" event={"ID":"dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1","Type":"ContainerDied","Data":"8fa161581bd1e2073bfb982b9fc5ef003db36d31412d2a9596906bbecb39ad9e"}
Jul 17 11:38:38.586322 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.586131    2566 scope.go:117] "RemoveContainer" containerID="89c03123c3ebe5be2d7199198210d555457a317d9ce36a759eb0ff639e77b381"
Jul 17 11:38:38.594699 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.594676    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce15a1d-ffb8-443a-80cd-e3e1fce97d57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk\" (UID: \"cce15a1d-ffb8-443a-80cd-e3e1fce97d57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"
Jul 17 11:38:38.609812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.609785    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"]
Jul 17 11:38:38.614093 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.614069    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"]
Jul 17 11:38:38.625516 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.625497    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6m2bx"]
Jul 17 11:38:38.696287 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.696255    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce15a1d-ffb8-443a-80cd-e3e1fce97d57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk\" (UID: \"cce15a1d-ffb8-443a-80cd-e3e1fce97d57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"
Jul 17 11:38:38.696912 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.696890    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce15a1d-ffb8-443a-80cd-e3e1fce97d57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk\" (UID: \"cce15a1d-ffb8-443a-80cd-e3e1fce97d57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"
Jul 17 11:38:38.743648 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.743625    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"
Jul 17 11:38:38.784092 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.784066    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"
Jul 17 11:38:38.898275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.898241    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f784e7ff-bf48-4937-8966-ffe0571c93cd-maas-api-service-ca\") pod \"f784e7ff-bf48-4937-8966-ffe0571c93cd\" (UID: \"f784e7ff-bf48-4937-8966-ffe0571c93cd\") "
Jul 17 11:38:38.898621 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.898597    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f784e7ff-bf48-4937-8966-ffe0571c93cd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f784e7ff-bf48-4937-8966-ffe0571c93cd" (UID: "f784e7ff-bf48-4937-8966-ffe0571c93cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:38:38.918482 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.918446    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"]
Jul 17 11:38:38.921270 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:38:38.921240    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce15a1d_ffb8_443a_80cd_e3e1fce97d57.slice/crio-c71b830340e59432f3ecd24ce1dde5638372aa9beee54a41c8d16e29fae45b0d WatchSource:0}: Error finding container c71b830340e59432f3ecd24ce1dde5638372aa9beee54a41c8d16e29fae45b0d: Status 404 returned error can't find the container with id c71b830340e59432f3ecd24ce1dde5638372aa9beee54a41c8d16e29fae45b0d
Jul 17 11:38:38.998929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:38.998866    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f784e7ff-bf48-4937-8966-ffe0571c93cd-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:38:39.594677 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.594651    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"
Jul 17 11:38:39.594831 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.594644    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk" event={"ID":"f784e7ff-bf48-4937-8966-ffe0571c93cd","Type":"ContainerDied","Data":"7090480f24b05bf2e0521e4d4ce7e40d96f6baead02966e51c610a3fe0716360"}
Jul 17 11:38:39.594831 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.594805    2566 scope.go:117] "RemoveContainer" containerID="7a03c80af615b8f13b49609eab3391005ae2f1e7ba17c5d8b42a73f4e963c13b"
Jul 17 11:38:39.596333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.596307    2566 generic.go:356] "Generic (PLEG): container finished" podID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerID="6b42a1fc5353ea224cf85e5d9fdefcc457bc6b3627b5ee8b97fb11a6b14c34d0" exitCode=6
Jul 17 11:38:39.596431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.596356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk" event={"ID":"cce15a1d-ffb8-443a-80cd-e3e1fce97d57","Type":"ContainerDied","Data":"6b42a1fc5353ea224cf85e5d9fdefcc457bc6b3627b5ee8b97fb11a6b14c34d0"}
Jul 17 11:38:39.596431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.596387    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk" event={"ID":"cce15a1d-ffb8-443a-80cd-e3e1fce97d57","Type":"ContainerStarted","Data":"c71b830340e59432f3ecd24ce1dde5638372aa9beee54a41c8d16e29fae45b0d"}
Jul 17 11:38:39.596576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.596562    2566 scope.go:117] "RemoveContainer" containerID="6b42a1fc5353ea224cf85e5d9fdefcc457bc6b3627b5ee8b97fb11a6b14c34d0"
Jul 17 11:38:39.657961 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.657937    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" path="/var/lib/kubelet/pods/dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1/volumes"
Jul 17 11:38:39.658218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.658206    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"]
Jul 17 11:38:39.663851 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:39.663832    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-jqpbk"]
Jul 17 11:38:40.605002 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:40.604911    2566 generic.go:356] "Generic (PLEG): container finished" podID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerID="7acd4f78d120fa3d30a39226fb85dbf360990aadea9045ae1c5baeac71e4649f" exitCode=6
Jul 17 11:38:40.605002 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:40.604984    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk" event={"ID":"cce15a1d-ffb8-443a-80cd-e3e1fce97d57","Type":"ContainerDied","Data":"7acd4f78d120fa3d30a39226fb85dbf360990aadea9045ae1c5baeac71e4649f"}
Jul 17 11:38:40.605482 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:40.605067    2566 scope.go:117] "RemoveContainer" containerID="6b42a1fc5353ea224cf85e5d9fdefcc457bc6b3627b5ee8b97fb11a6b14c34d0"
Jul 17 11:38:40.605482 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:40.605224    2566 scope.go:117] "RemoveContainer" containerID="7acd4f78d120fa3d30a39226fb85dbf360990aadea9045ae1c5baeac71e4649f"
Jul 17 11:38:40.605482 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:40.605469    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk_odh-ai-gateway-infra(cce15a1d-ffb8-443a-80cd-e3e1fce97d57)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57"
Jul 17 11:38:41.612105 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:41.612077    2566 scope.go:117] "RemoveContainer" containerID="7acd4f78d120fa3d30a39226fb85dbf360990aadea9045ae1c5baeac71e4649f"
Jul 17 11:38:41.612476 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:41.612309    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk_odh-ai-gateway-infra(cce15a1d-ffb8-443a-80cd-e3e1fce97d57)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57"
Jul 17 11:38:41.657341 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:41.657309    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" path="/var/lib/kubelet/pods/f784e7ff-bf48-4937-8966-ffe0571c93cd/volumes"
Jul 17 11:38:55.260632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.260547    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"]
Jul 17 11:38:55.261188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.261145    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerName="revoke-keys"
Jul 17 11:38:55.261188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.261168    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerName="revoke-keys"
Jul 17 11:38:55.261188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.261184    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerName="revoke-keys"
Jul 17 11:38:55.261369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.261193    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerName="revoke-keys"
Jul 17 11:38:55.261369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.261211    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerName="revoke-keys"
Jul 17 11:38:55.261369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.261219    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerName="revoke-keys"
Jul 17 11:38:55.261369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.261314    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerName="revoke-keys"
Jul 17 11:38:55.261369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.261325    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerName="revoke-keys"
Jul 17 11:38:55.261369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.261337    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd8fcc1c-ba76-4333-8dab-7b6fa61f11c1" containerName="revoke-keys"
Jul 17 11:38:55.265898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.265875    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"
Jul 17 11:38:55.275059 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.275036    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"]
Jul 17 11:38:55.342059 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.342025    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320fbcd3-7a77-4b6c-a476-d06e84a736dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-rtssf\" (UID: \"320fbcd3-7a77-4b6c-a476-d06e84a736dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"
Jul 17 11:38:55.442839 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.442808    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320fbcd3-7a77-4b6c-a476-d06e84a736dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-rtssf\" (UID: \"320fbcd3-7a77-4b6c-a476-d06e84a736dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"
Jul 17 11:38:55.443351 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.443331    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320fbcd3-7a77-4b6c-a476-d06e84a736dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-rtssf\" (UID: \"320fbcd3-7a77-4b6c-a476-d06e84a736dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"
Jul 17 11:38:55.578197 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.578174    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"
Jul 17 11:38:55.709656 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:55.709632    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"]
Jul 17 11:38:55.711861 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:38:55.711831    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320fbcd3_7a77_4b6c_a476_d06e84a736dd.slice/crio-eb78205ad23fddd01e0881792cb9700a2f17270c63c19256e75b0a2c96cf3eef WatchSource:0}: Error finding container eb78205ad23fddd01e0881792cb9700a2f17270c63c19256e75b0a2c96cf3eef: Status 404 returned error can't find the container with id eb78205ad23fddd01e0881792cb9700a2f17270c63c19256e75b0a2c96cf3eef
Jul 17 11:38:56.651983 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:56.651904    2566 scope.go:117] "RemoveContainer" containerID="7acd4f78d120fa3d30a39226fb85dbf360990aadea9045ae1c5baeac71e4649f"
Jul 17 11:38:56.686772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:56.686743    2566 generic.go:356] "Generic (PLEG): container finished" podID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerID="8a311d4bece4bd06f0e7c343dd3f503aa3f9ad11d1534bdd704e767d73a7c701" exitCode=6
Jul 17 11:38:56.686874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:56.686825    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf" event={"ID":"320fbcd3-7a77-4b6c-a476-d06e84a736dd","Type":"ContainerDied","Data":"8a311d4bece4bd06f0e7c343dd3f503aa3f9ad11d1534bdd704e767d73a7c701"}
Jul 17 11:38:56.686874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:56.686872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf" event={"ID":"320fbcd3-7a77-4b6c-a476-d06e84a736dd","Type":"ContainerStarted","Data":"eb78205ad23fddd01e0881792cb9700a2f17270c63c19256e75b0a2c96cf3eef"}
Jul 17 11:38:56.686985 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:56.686946    2566 scope.go:117] "RemoveContainer" containerID="8a311d4bece4bd06f0e7c343dd3f503aa3f9ad11d1534bdd704e767d73a7c701"
Jul 17 11:38:57.693127 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:57.693095    2566 generic.go:356] "Generic (PLEG): container finished" podID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerID="3bccb56509c35effc95a7bfb6dcfcd326e0cce076e430721332aab3d662365d0" exitCode=6
Jul 17 11:38:57.693553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:57.693168    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk" event={"ID":"cce15a1d-ffb8-443a-80cd-e3e1fce97d57","Type":"ContainerDied","Data":"3bccb56509c35effc95a7bfb6dcfcd326e0cce076e430721332aab3d662365d0"}
Jul 17 11:38:57.693553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:57.693222    2566 scope.go:117] "RemoveContainer" containerID="7acd4f78d120fa3d30a39226fb85dbf360990aadea9045ae1c5baeac71e4649f"
Jul 17 11:38:57.693553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:57.693528    2566 scope.go:117] "RemoveContainer" containerID="3bccb56509c35effc95a7bfb6dcfcd326e0cce076e430721332aab3d662365d0"
Jul 17 11:38:57.693803 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:57.693778    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk_odh-ai-gateway-infra(cce15a1d-ffb8-443a-80cd-e3e1fce97d57)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57"
Jul 17 11:38:57.695045 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:57.695022    2566 generic.go:356] "Generic (PLEG): container finished" podID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerID="9797df5f9ffb78a923282e8bda965036821c12d5d30dc278dccd6776c86dfef7" exitCode=6
Jul 17 11:38:57.695147 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:57.695099    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf" event={"ID":"320fbcd3-7a77-4b6c-a476-d06e84a736dd","Type":"ContainerDied","Data":"9797df5f9ffb78a923282e8bda965036821c12d5d30dc278dccd6776c86dfef7"}
Jul 17 11:38:57.695253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:57.695238    2566 scope.go:117] "RemoveContainer" containerID="9797df5f9ffb78a923282e8bda965036821c12d5d30dc278dccd6776c86dfef7"
Jul 17 11:38:57.695439 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:57.695421    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-rtssf_odh-ai-gateway-infra(320fbcd3-7a77-4b6c-a476-d06e84a736dd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd"
Jul 17 11:38:57.704080 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:57.704064    2566 scope.go:117] "RemoveContainer" containerID="8a311d4bece4bd06f0e7c343dd3f503aa3f9ad11d1534bdd704e767d73a7c701"
Jul 17 11:38:58.703070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:58.703044    2566 scope.go:117] "RemoveContainer" containerID="9797df5f9ffb78a923282e8bda965036821c12d5d30dc278dccd6776c86dfef7"
Jul 17 11:38:58.703420 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:38:58.703225    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-rtssf_odh-ai-gateway-infra(320fbcd3-7a77-4b6c-a476-d06e84a736dd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd"
Jul 17 11:38:58.735180 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:58.735153    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"]
Jul 17 11:38:58.863242 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:58.863221    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"
Jul 17 11:38:58.974842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:58.974762    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce15a1d-ffb8-443a-80cd-e3e1fce97d57-maas-api-service-ca\") pod \"cce15a1d-ffb8-443a-80cd-e3e1fce97d57\" (UID: \"cce15a1d-ffb8-443a-80cd-e3e1fce97d57\") "
Jul 17 11:38:58.975148 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:58.975119    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cce15a1d-ffb8-443a-80cd-e3e1fce97d57-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cce15a1d-ffb8-443a-80cd-e3e1fce97d57" (UID: "cce15a1d-ffb8-443a-80cd-e3e1fce97d57"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:38:59.075801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:59.075775    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce15a1d-ffb8-443a-80cd-e3e1fce97d57-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:38:59.709439 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:59.709413    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"
Jul 17 11:38:59.709808 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:59.709445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk" event={"ID":"cce15a1d-ffb8-443a-80cd-e3e1fce97d57","Type":"ContainerDied","Data":"c71b830340e59432f3ecd24ce1dde5638372aa9beee54a41c8d16e29fae45b0d"}
Jul 17 11:38:59.709808 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:59.709506    2566 scope.go:117] "RemoveContainer" containerID="3bccb56509c35effc95a7bfb6dcfcd326e0cce076e430721332aab3d662365d0"
Jul 17 11:38:59.730209 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:59.730187    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"]
Jul 17 11:38:59.738840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:38:59.738819    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-s5zdk"]
Jul 17 11:39:01.657376 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:01.657344    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" path="/var/lib/kubelet/pods/cce15a1d-ffb8-443a-80cd-e3e1fce97d57/volumes"
Jul 17 11:39:09.652584 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:09.652544    2566 scope.go:117] "RemoveContainer" containerID="9797df5f9ffb78a923282e8bda965036821c12d5d30dc278dccd6776c86dfef7"
Jul 17 11:39:10.762545 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:10.762513    2566 generic.go:356] "Generic (PLEG): container finished" podID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerID="6f60c4cab06f43e6fb9bc290ee174f3a9bb8202b87f1f4e8ee90f7798b57cf95" exitCode=6
Jul 17 11:39:10.762974 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:10.762590    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf" event={"ID":"320fbcd3-7a77-4b6c-a476-d06e84a736dd","Type":"ContainerDied","Data":"6f60c4cab06f43e6fb9bc290ee174f3a9bb8202b87f1f4e8ee90f7798b57cf95"}
Jul 17 11:39:10.762974 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:10.762664    2566 scope.go:117] "RemoveContainer" containerID="9797df5f9ffb78a923282e8bda965036821c12d5d30dc278dccd6776c86dfef7"
Jul 17 11:39:10.762974 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:10.762911    2566 scope.go:117] "RemoveContainer" containerID="6f60c4cab06f43e6fb9bc290ee174f3a9bb8202b87f1f4e8ee90f7798b57cf95"
Jul 17 11:39:10.763159 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:10.763140    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-rtssf_odh-ai-gateway-infra(320fbcd3-7a77-4b6c-a476-d06e84a736dd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd"
Jul 17 11:39:11.798178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:11.798146    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"]
Jul 17 11:39:11.932118 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:11.932090    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"
Jul 17 11:39:12.084664 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:12.084635    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320fbcd3-7a77-4b6c-a476-d06e84a736dd-maas-api-service-ca\") pod \"320fbcd3-7a77-4b6c-a476-d06e84a736dd\" (UID: \"320fbcd3-7a77-4b6c-a476-d06e84a736dd\") "
Jul 17 11:39:12.084986 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:12.084964    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/320fbcd3-7a77-4b6c-a476-d06e84a736dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "320fbcd3-7a77-4b6c-a476-d06e84a736dd" (UID: "320fbcd3-7a77-4b6c-a476-d06e84a736dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:39:12.185605 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:12.185578    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320fbcd3-7a77-4b6c-a476-d06e84a736dd-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:39:12.774253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:12.774224    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"
Jul 17 11:39:12.774422 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:12.774221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf" event={"ID":"320fbcd3-7a77-4b6c-a476-d06e84a736dd","Type":"ContainerDied","Data":"eb78205ad23fddd01e0881792cb9700a2f17270c63c19256e75b0a2c96cf3eef"}
Jul 17 11:39:12.774422 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:12.774357    2566 scope.go:117] "RemoveContainer" containerID="6f60c4cab06f43e6fb9bc290ee174f3a9bb8202b87f1f4e8ee90f7798b57cf95"
Jul 17 11:39:12.804345 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:12.804321    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"]
Jul 17 11:39:12.808894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:12.808871    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-rtssf"]
Jul 17 11:39:13.657603 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:13.657570    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" path="/var/lib/kubelet/pods/320fbcd3-7a77-4b6c-a476-d06e84a736dd/volumes"
Jul 17 11:39:15.744171 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:15.744134    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.377470    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"]
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378428    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378450    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378478    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378488    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378510    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378519    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378529    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378537    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378564    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378572    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378588    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378596    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378773    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378784    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378802    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f784e7ff-bf48-4937-8966-ffe0571c93cd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378811    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378827    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378837    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerName="revoke-keys"
Jul 17 11:39:17.380901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.378847    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce15a1d-ffb8-443a-80cd-e3e1fce97d57" containerName="revoke-keys"
Jul 17 11:39:17.384494 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.384466    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"
Jul 17 11:39:17.387752 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.387695    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:39:17.387891 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.387695    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:39:17.391091 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.391069    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"]
Jul 17 11:39:17.529753 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.529695    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d3f6d0-cd9b-4f08-b238-02492dbc3332-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-r2fvk\" (UID: \"93d3f6d0-cd9b-4f08-b238-02492dbc3332\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"
Jul 17 11:39:17.630132 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.630103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d3f6d0-cd9b-4f08-b238-02492dbc3332-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-r2fvk\" (UID: \"93d3f6d0-cd9b-4f08-b238-02492dbc3332\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"
Jul 17 11:39:17.630762 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.630738    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d3f6d0-cd9b-4f08-b238-02492dbc3332-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-r2fvk\" (UID: \"93d3f6d0-cd9b-4f08-b238-02492dbc3332\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"
Jul 17 11:39:17.696628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.696572    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"
Jul 17 11:39:17.828609 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:17.828581    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"]
Jul 17 11:39:17.830372 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:39:17.830340    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d3f6d0_cd9b_4f08_b238_02492dbc3332.slice/crio-a9dbeedf3075a79324746915696f7309bca937e64409ee37454df68358602123 WatchSource:0}: Error finding container a9dbeedf3075a79324746915696f7309bca937e64409ee37454df68358602123: Status 404 returned error can't find the container with id a9dbeedf3075a79324746915696f7309bca937e64409ee37454df68358602123
Jul 17 11:39:18.592549 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.592516    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"]
Jul 17 11:39:18.592942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.592879    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerName="revoke-keys"
Jul 17 11:39:18.592942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.592891    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="320fbcd3-7a77-4b6c-a476-d06e84a736dd" containerName="revoke-keys"
Jul 17 11:39:18.595888 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.595872    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"
Jul 17 11:39:18.605238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.605215    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"]
Jul 17 11:39:18.738234 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.738212    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ecfa2b-1943-4cce-862e-ae6bf1020cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8\" (UID: \"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"
Jul 17 11:39:18.806557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.806529    2566 generic.go:356] "Generic (PLEG): container finished" podID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerID="cc412ddad2d574324eed514d8c683d232c5b356c824816bf5febedc68cbd3b58" exitCode=6
Jul 17 11:39:18.806660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.806600    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" event={"ID":"93d3f6d0-cd9b-4f08-b238-02492dbc3332","Type":"ContainerDied","Data":"cc412ddad2d574324eed514d8c683d232c5b356c824816bf5febedc68cbd3b58"}
Jul 17 11:39:18.806660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.806635    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" event={"ID":"93d3f6d0-cd9b-4f08-b238-02492dbc3332","Type":"ContainerStarted","Data":"a9dbeedf3075a79324746915696f7309bca937e64409ee37454df68358602123"}
Jul 17 11:39:18.806805 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.806789    2566 scope.go:117] "RemoveContainer" containerID="cc412ddad2d574324eed514d8c683d232c5b356c824816bf5febedc68cbd3b58"
Jul 17 11:39:18.839557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.839530    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ecfa2b-1943-4cce-862e-ae6bf1020cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8\" (UID: \"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"
Jul 17 11:39:18.840149 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.840132    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ecfa2b-1943-4cce-862e-ae6bf1020cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8\" (UID: \"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"
Jul 17 11:39:18.906642 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:18.906563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"
Jul 17 11:39:19.245633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:19.245610    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"]
Jul 17 11:39:19.247591 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:39:19.247566    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ecfa2b_1943_4cce_862e_ae6bf1020cd9.slice/crio-64aeed10c674f35f722f8bda0366106a3a5100750155189021a026b97e05023d WatchSource:0}: Error finding container 64aeed10c674f35f722f8bda0366106a3a5100750155189021a026b97e05023d: Status 404 returned error can't find the container with id 64aeed10c674f35f722f8bda0366106a3a5100750155189021a026b97e05023d
Jul 17 11:39:19.812600 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:19.812500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" event={"ID":"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9","Type":"ContainerStarted","Data":"3e6a71f3b6caa69f1fb087236ecc93bb34f1799bb7182f7a1af87ebb65b717ab"}
Jul 17 11:39:19.812600 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:19.812556    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" event={"ID":"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9","Type":"ContainerStarted","Data":"64aeed10c674f35f722f8bda0366106a3a5100750155189021a026b97e05023d"}
Jul 17 11:39:19.814233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:19.814207    2566 generic.go:356] "Generic (PLEG): container finished" podID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerID="8ea0b811a00cb7b25b8d15a68ba891ae36b37369cef4dd8c7efba908d728bfff" exitCode=6
Jul 17 11:39:19.814345 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:19.814256    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" event={"ID":"93d3f6d0-cd9b-4f08-b238-02492dbc3332","Type":"ContainerDied","Data":"8ea0b811a00cb7b25b8d15a68ba891ae36b37369cef4dd8c7efba908d728bfff"}
Jul 17 11:39:19.814345 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:19.814302    2566 scope.go:117] "RemoveContainer" containerID="cc412ddad2d574324eed514d8c683d232c5b356c824816bf5febedc68cbd3b58"
Jul 17 11:39:19.814490 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:19.814448    2566 scope.go:117] "RemoveContainer" containerID="8ea0b811a00cb7b25b8d15a68ba891ae36b37369cef4dd8c7efba908d728bfff"
Jul 17 11:39:19.814694 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:19.814671    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-r2fvk_odh-ai-gateway-infra(93d3f6d0-cd9b-4f08-b238-02492dbc3332)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332"
Jul 17 11:39:19.839903 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:19.839862    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" podStartSLOduration=1.839846222 podStartE2EDuration="1.839846222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:39:19.838748629 +0000 UTC m=+2542.695832575" watchObservedRunningTime="2026-07-17 11:39:19.839846222 +0000 UTC m=+2542.696930169"
Jul 17 11:39:20.820406 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:20.820364    2566 scope.go:117] "RemoveContainer" containerID="8ea0b811a00cb7b25b8d15a68ba891ae36b37369cef4dd8c7efba908d728bfff"
Jul 17 11:39:20.820839 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:20.820590    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-r2fvk_odh-ai-gateway-infra(93d3f6d0-cd9b-4f08-b238-02492dbc3332)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332"
Jul 17 11:39:20.821671 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:20.821647    2566 generic.go:356] "Generic (PLEG): container finished" podID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerID="3e6a71f3b6caa69f1fb087236ecc93bb34f1799bb7182f7a1af87ebb65b717ab" exitCode=6
Jul 17 11:39:20.821754 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:20.821700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" event={"ID":"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9","Type":"ContainerDied","Data":"3e6a71f3b6caa69f1fb087236ecc93bb34f1799bb7182f7a1af87ebb65b717ab"}
Jul 17 11:39:20.821902 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:20.821888    2566 scope.go:117] "RemoveContainer" containerID="3e6a71f3b6caa69f1fb087236ecc93bb34f1799bb7182f7a1af87ebb65b717ab"
Jul 17 11:39:21.827799 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:21.827768    2566 generic.go:356] "Generic (PLEG): container finished" podID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerID="a46aa3c110835b5ac0ff379d7bf4eb670b90a6b2fe15d99346f620606d418eea" exitCode=6
Jul 17 11:39:21.828151 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:21.827855    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" event={"ID":"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9","Type":"ContainerDied","Data":"a46aa3c110835b5ac0ff379d7bf4eb670b90a6b2fe15d99346f620606d418eea"}
Jul 17 11:39:21.828151 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:21.827959    2566 scope.go:117] "RemoveContainer" containerID="3e6a71f3b6caa69f1fb087236ecc93bb34f1799bb7182f7a1af87ebb65b717ab"
Jul 17 11:39:21.828151 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:21.828045    2566 scope.go:117] "RemoveContainer" containerID="a46aa3c110835b5ac0ff379d7bf4eb670b90a6b2fe15d99346f620606d418eea"
Jul 17 11:39:21.828283 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:21.828248    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8_odh-ai-gateway-infra(e4ecfa2b-1943-4cce-862e-ae6bf1020cd9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9"
Jul 17 11:39:22.836716 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:22.836686    2566 scope.go:117] "RemoveContainer" containerID="a46aa3c110835b5ac0ff379d7bf4eb670b90a6b2fe15d99346f620606d418eea"
Jul 17 11:39:22.837095 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:22.836922    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8_odh-ai-gateway-infra(e4ecfa2b-1943-4cce-862e-ae6bf1020cd9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9"
Jul 17 11:39:23.266661 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:23.266569    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:39:30.488400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.488367    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"]
Jul 17 11:39:30.493449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.493424    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.496117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.496095    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5a6d7b05-openshift-default-dockercfg-zjlq8\""
Jul 17 11:39:30.504542 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.504515    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"]
Jul 17 11:39:30.635427 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.635396    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03166279-6a7f-4735-82b4-01335014fe06-istiod-ca-cert\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.635602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.635436    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-credential-socket\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.635602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.635454    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03166279-6a7f-4735-82b4-01335014fe06-istio-podinfo\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.635602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.635540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-data\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.635602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.635580    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-istio-token\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.635770 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.635620    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-socket\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.635770 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.635639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-envoy\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.635770 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.635690    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlkrl\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-kube-api-access-tlkrl\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.635770 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.635762    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-certs\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.736988 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.736954    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-istio-token\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737150 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737003    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-socket\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737150 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-envoy\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737150 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737048    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlkrl\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-kube-api-access-tlkrl\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737150 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737087    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-certs\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737150 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03166279-6a7f-4735-82b4-01335014fe06-istiod-ca-cert\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737407 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737150    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-credential-socket\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737407 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737339    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03166279-6a7f-4735-82b4-01335014fe06-istio-podinfo\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737407 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737397    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-data\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737566 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737414    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-socket\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737566 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737542    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-certs\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737627    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-credential-socket\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737774    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-data\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.737883 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.737859    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03166279-6a7f-4735-82b4-01335014fe06-istiod-ca-cert\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.739414 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.739360    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-envoy\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.739646 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.739629    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03166279-6a7f-4735-82b4-01335014fe06-istio-podinfo\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.746942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.746918    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlkrl\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-kube-api-access-tlkrl\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.748517 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.748491    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-istio-token\") pod \"e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") " pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.804217 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.804181    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:30.847632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.847598    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:39:30.944967 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.944935    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"]
Jul 17 11:39:30.948111 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:39:30.948086    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03166279_6a7f_4735_82b4_01335014fe06.slice/crio-95e12eeffedbfb5eb5e91454e953aa26b046653f552c2604814fc6f15c521934 WatchSource:0}: Error finding container 95e12eeffedbfb5eb5e91454e953aa26b046653f552c2604814fc6f15c521934: Status 404 returned error can't find the container with id 95e12eeffedbfb5eb5e91454e953aa26b046653f552c2604814fc6f15c521934
Jul 17 11:39:30.950149 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.950112    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:39:30.950237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.950182    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:39:30.950237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:30.950211    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:39:31.882513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:31.882480    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" event={"ID":"03166279-6a7f-4735-82b4-01335014fe06","Type":"ContainerStarted","Data":"61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037"}
Jul 17 11:39:31.882944 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:31.882534    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" event={"ID":"03166279-6a7f-4735-82b4-01335014fe06","Type":"ContainerStarted","Data":"95e12eeffedbfb5eb5e91454e953aa26b046653f552c2604814fc6f15c521934"}
Jul 17 11:39:31.953833 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:31.953786    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" podStartSLOduration=1.953770926 podStartE2EDuration="1.953770926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:39:31.951892925 +0000 UTC m=+2554.808976869" watchObservedRunningTime="2026-07-17 11:39:31.953770926 +0000 UTC m=+2554.810854870"
Jul 17 11:39:32.651584 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:32.651557    2566 scope.go:117] "RemoveContainer" containerID="8ea0b811a00cb7b25b8d15a68ba891ae36b37369cef4dd8c7efba908d728bfff"
Jul 17 11:39:32.804925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:32.804890    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:32.809915 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:32.809892    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:32.888225 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:32.888189    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" event={"ID":"93d3f6d0-cd9b-4f08-b238-02492dbc3332","Type":"ContainerStarted","Data":"2717e6ea111f2cf2376a62d67bdbe410f0464590e80c9e37829b5aff4d05636a"}
Jul 17 11:39:32.888563 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:32.888545    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:32.889447 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:32.889429    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:32.909697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:32.909619    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" podStartSLOduration=15.909605379 podStartE2EDuration="15.909605379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:39:32.907780921 +0000 UTC m=+2555.764864867" watchObservedRunningTime="2026-07-17 11:39:32.909605379 +0000 UTC m=+2555.766689324"
Jul 17 11:39:33.895009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:33.894973    2566 generic.go:356] "Generic (PLEG): container finished" podID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerID="2717e6ea111f2cf2376a62d67bdbe410f0464590e80c9e37829b5aff4d05636a" exitCode=6
Jul 17 11:39:33.897549 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:33.895045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" event={"ID":"93d3f6d0-cd9b-4f08-b238-02492dbc3332","Type":"ContainerDied","Data":"2717e6ea111f2cf2376a62d67bdbe410f0464590e80c9e37829b5aff4d05636a"}
Jul 17 11:39:33.897549 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:33.895101    2566 scope.go:117] "RemoveContainer" containerID="8ea0b811a00cb7b25b8d15a68ba891ae36b37369cef4dd8c7efba908d728bfff"
Jul 17 11:39:33.897549 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:33.895511    2566 scope.go:117] "RemoveContainer" containerID="2717e6ea111f2cf2376a62d67bdbe410f0464590e80c9e37829b5aff4d05636a"
Jul 17 11:39:33.897549 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:33.895733    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-r2fvk_odh-ai-gateway-infra(93d3f6d0-cd9b-4f08-b238-02492dbc3332)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332"
Jul 17 11:39:33.925257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:33.925223    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"]
Jul 17 11:39:35.041733 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.041685    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"
Jul 17 11:39:35.076927 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.076899    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d3f6d0-cd9b-4f08-b238-02492dbc3332-maas-api-service-ca\") pod \"93d3f6d0-cd9b-4f08-b238-02492dbc3332\" (UID: \"93d3f6d0-cd9b-4f08-b238-02492dbc3332\") "
Jul 17 11:39:35.077229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.077205    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93d3f6d0-cd9b-4f08-b238-02492dbc3332-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "93d3f6d0-cd9b-4f08-b238-02492dbc3332" (UID: "93d3f6d0-cd9b-4f08-b238-02492dbc3332"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:39:35.177509 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.177431    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d3f6d0-cd9b-4f08-b238-02492dbc3332-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:39:35.651602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.651576    2566 scope.go:117] "RemoveContainer" containerID="a46aa3c110835b5ac0ff379d7bf4eb670b90a6b2fe15d99346f620606d418eea"
Jul 17 11:39:35.907940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.907858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk" event={"ID":"93d3f6d0-cd9b-4f08-b238-02492dbc3332","Type":"ContainerDied","Data":"a9dbeedf3075a79324746915696f7309bca937e64409ee37454df68358602123"}
Jul 17 11:39:35.907940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.907925    2566 scope.go:117] "RemoveContainer" containerID="2717e6ea111f2cf2376a62d67bdbe410f0464590e80c9e37829b5aff4d05636a"
Jul 17 11:39:35.908136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.907878    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"
Jul 17 11:39:35.927645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.927618    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"]
Jul 17 11:39:35.933623 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:35.933601    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-r2fvk"]
Jul 17 11:39:36.915075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:36.915046    2566 generic.go:356] "Generic (PLEG): container finished" podID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerID="f3a618de4892ecd2cb6313abd8377308adf1158c47ba8516a660433344f1a88f" exitCode=6
Jul 17 11:39:36.915510 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:36.915126    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" event={"ID":"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9","Type":"ContainerDied","Data":"f3a618de4892ecd2cb6313abd8377308adf1158c47ba8516a660433344f1a88f"}
Jul 17 11:39:36.915510 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:36.915188    2566 scope.go:117] "RemoveContainer" containerID="a46aa3c110835b5ac0ff379d7bf4eb670b90a6b2fe15d99346f620606d418eea"
Jul 17 11:39:36.915510 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:36.915460    2566 scope.go:117] "RemoveContainer" containerID="f3a618de4892ecd2cb6313abd8377308adf1158c47ba8516a660433344f1a88f"
Jul 17 11:39:36.915717 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:36.915679    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8_odh-ai-gateway-infra(e4ecfa2b-1943-4cce-862e-ae6bf1020cd9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9"
Jul 17 11:39:37.657601 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:37.657566    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" path="/var/lib/kubelet/pods/93d3f6d0-cd9b-4f08-b238-02492dbc3332/volumes"
Jul 17 11:39:37.947803 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:37.947716    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"]
Jul 17 11:39:38.080959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.080928    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"
Jul 17 11:39:38.096857 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.096824    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ecfa2b-1943-4cce-862e-ae6bf1020cd9-maas-api-service-ca\") pod \"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9\" (UID: \"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9\") "
Jul 17 11:39:38.097154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.097133    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ecfa2b-1943-4cce-862e-ae6bf1020cd9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" (UID: "e4ecfa2b-1943-4cce-862e-ae6bf1020cd9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:39:38.197670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.197634    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ecfa2b-1943-4cce-862e-ae6bf1020cd9-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:39:38.530322 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530288    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"]
Jul 17 11:39:38.530722 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530694    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerName="revoke-keys"
Jul 17 11:39:38.530722 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530720    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530735    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530744    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530758    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530763    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530774    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530779    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530786    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530791    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530799    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerName="revoke-keys"
Jul 17 11:39:38.530801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530805    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerName="revoke-keys"
Jul 17 11:39:38.531099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530861    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerName="revoke-keys"
Jul 17 11:39:38.531099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530869    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerName="revoke-keys"
Jul 17 11:39:38.531099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530875    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerName="revoke-keys"
Jul 17 11:39:38.531099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.530882    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerName="revoke-keys"
Jul 17 11:39:38.533916 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.533900    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"
Jul 17 11:39:38.543980 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.543951    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"]
Jul 17 11:39:38.601611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.601576    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdd7fde-1d32-440a-ba62-82f1410726af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c\" (UID: \"fcdd7fde-1d32-440a-ba62-82f1410726af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"
Jul 17 11:39:38.703150 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.703117    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdd7fde-1d32-440a-ba62-82f1410726af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c\" (UID: \"fcdd7fde-1d32-440a-ba62-82f1410726af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"
Jul 17 11:39:38.703857 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.703834    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdd7fde-1d32-440a-ba62-82f1410726af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c\" (UID: \"fcdd7fde-1d32-440a-ba62-82f1410726af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"
Jul 17 11:39:38.756092 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.756062    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:39:38.846243 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.846216    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"
Jul 17 11:39:38.928055 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.928029    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"
Jul 17 11:39:38.928193 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.928021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8" event={"ID":"e4ecfa2b-1943-4cce-862e-ae6bf1020cd9","Type":"ContainerDied","Data":"64aeed10c674f35f722f8bda0366106a3a5100750155189021a026b97e05023d"}
Jul 17 11:39:38.928268 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.928201    2566 scope.go:117] "RemoveContainer" containerID="f3a618de4892ecd2cb6313abd8377308adf1158c47ba8516a660433344f1a88f"
Jul 17 11:39:38.962491 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.962454    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"]
Jul 17 11:39:38.967543 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.967508    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-8mfj8"]
Jul 17 11:39:38.987783 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:38.987759    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"]
Jul 17 11:39:38.989831 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:39:38.989808    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcdd7fde_1d32_440a_ba62_82f1410726af.slice/crio-1285f8da079e78411299feebce1c07f2b372e1ba2ace018c5b854aa00bd67094 WatchSource:0}: Error finding container 1285f8da079e78411299feebce1c07f2b372e1ba2ace018c5b854aa00bd67094: Status 404 returned error can't find the container with id 1285f8da079e78411299feebce1c07f2b372e1ba2ace018c5b854aa00bd67094
Jul 17 11:39:39.657528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:39.657497    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" path="/var/lib/kubelet/pods/e4ecfa2b-1943-4cce-862e-ae6bf1020cd9/volumes"
Jul 17 11:39:39.935394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:39.935292    2566 generic.go:356] "Generic (PLEG): container finished" podID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerID="138590e0c902cfa4373146d7562f004ab9ac0aabf8628be8e950f95c9b918b7a" exitCode=6
Jul 17 11:39:39.935394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:39.935337    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" event={"ID":"fcdd7fde-1d32-440a-ba62-82f1410726af","Type":"ContainerDied","Data":"138590e0c902cfa4373146d7562f004ab9ac0aabf8628be8e950f95c9b918b7a"}
Jul 17 11:39:39.935629 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:39.935418    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" event={"ID":"fcdd7fde-1d32-440a-ba62-82f1410726af","Type":"ContainerStarted","Data":"1285f8da079e78411299feebce1c07f2b372e1ba2ace018c5b854aa00bd67094"}
Jul 17 11:39:39.935670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:39.935658    2566 scope.go:117] "RemoveContainer" containerID="138590e0c902cfa4373146d7562f004ab9ac0aabf8628be8e950f95c9b918b7a"
Jul 17 11:39:40.947561 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:40.946034    2566 generic.go:356] "Generic (PLEG): container finished" podID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerID="2fadb0a404a0b5f9c13ea432c90ecec34035406da47147c6165d0809500fd6ca" exitCode=6
Jul 17 11:39:40.947561 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:40.946177    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" event={"ID":"fcdd7fde-1d32-440a-ba62-82f1410726af","Type":"ContainerDied","Data":"2fadb0a404a0b5f9c13ea432c90ecec34035406da47147c6165d0809500fd6ca"}
Jul 17 11:39:40.947561 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:40.946235    2566 scope.go:117] "RemoveContainer" containerID="138590e0c902cfa4373146d7562f004ab9ac0aabf8628be8e950f95c9b918b7a"
Jul 17 11:39:40.947561 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:40.946607    2566 scope.go:117] "RemoveContainer" containerID="2fadb0a404a0b5f9c13ea432c90ecec34035406da47147c6165d0809500fd6ca"
Jul 17 11:39:40.947561 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:40.946897    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c_odh-ai-gateway-infra(fcdd7fde-1d32-440a-ba62-82f1410726af)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af"
Jul 17 11:39:41.955661 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:41.955633    2566 scope.go:117] "RemoveContainer" containerID="2fadb0a404a0b5f9c13ea432c90ecec34035406da47147c6165d0809500fd6ca"
Jul 17 11:39:41.956071 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:41.955930    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c_odh-ai-gateway-infra(fcdd7fde-1d32-440a-ba62-82f1410726af)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af"
Jul 17 11:39:46.431849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.431810    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"]
Jul 17 11:39:46.432252 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.432239    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ecfa2b-1943-4cce-862e-ae6bf1020cd9" containerName="revoke-keys"
Jul 17 11:39:46.432293 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.432255    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="93d3f6d0-cd9b-4f08-b238-02492dbc3332" containerName="revoke-keys"
Jul 17 11:39:46.437416 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.437399    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"
Jul 17 11:39:46.444297 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.444272    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"]
Jul 17 11:39:46.488167 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.488130    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9322e6-d2c9-46e4-a964-f1902ec2d66c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85\" (UID: \"3f9322e6-d2c9-46e4-a964-f1902ec2d66c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"
Jul 17 11:39:46.589580 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.589539    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9322e6-d2c9-46e4-a964-f1902ec2d66c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85\" (UID: \"3f9322e6-d2c9-46e4-a964-f1902ec2d66c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"
Jul 17 11:39:46.590155 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.590129    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9322e6-d2c9-46e4-a964-f1902ec2d66c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85\" (UID: \"3f9322e6-d2c9-46e4-a964-f1902ec2d66c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"
Jul 17 11:39:46.749376 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.749309    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"
Jul 17 11:39:46.981461 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.981437    2566 generic.go:356] "Generic (PLEG): container finished" podID="03166279-6a7f-4735-82b4-01335014fe06" containerID="61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037" exitCode=0
Jul 17 11:39:46.981614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.981518    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" event={"ID":"03166279-6a7f-4735-82b4-01335014fe06","Type":"ContainerDied","Data":"61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037"}
Jul 17 11:39:46.981917 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.981903    2566 scope.go:117] "RemoveContainer" containerID="61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037"
Jul 17 11:39:46.982884 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.982855    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:39:46.982983 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.982940    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:39:46.983046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:46.982991    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:39:47.090781 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:47.090759    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"]
Jul 17 11:39:47.093530 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:39:47.093490    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f9322e6_d2c9_46e4_a964_f1902ec2d66c.slice/crio-af0d01376d6e4f1bcbf54a722b4c8a482fc33d5f883805c654fba900c0055c03 WatchSource:0}: Error finding container af0d01376d6e4f1bcbf54a722b4c8a482fc33d5f883805c654fba900c0055c03: Status 404 returned error can't find the container with id af0d01376d6e4f1bcbf54a722b4c8a482fc33d5f883805c654fba900c0055c03
Jul 17 11:39:47.888757 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:47.888726    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:47.988586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:47.988548    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" event={"ID":"03166279-6a7f-4735-82b4-01335014fe06","Type":"ContainerStarted","Data":"b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892"}
Jul 17 11:39:47.990033 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:47.990009    2566 generic.go:356] "Generic (PLEG): container finished" podID="3f9322e6-d2c9-46e4-a964-f1902ec2d66c" containerID="094c891843f86bc89c018acc011fa34a3f9d6c1832356d34fd8230146a9be68f" exitCode=0
Jul 17 11:39:47.990184 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:47.990094    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85" event={"ID":"3f9322e6-d2c9-46e4-a964-f1902ec2d66c","Type":"ContainerDied","Data":"094c891843f86bc89c018acc011fa34a3f9d6c1832356d34fd8230146a9be68f"}
Jul 17 11:39:47.990184 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:47.990149    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85" event={"ID":"3f9322e6-d2c9-46e4-a964-f1902ec2d66c","Type":"ContainerStarted","Data":"af0d01376d6e4f1bcbf54a722b4c8a482fc33d5f883805c654fba900c0055c03"}
Jul 17 11:39:48.804753 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:48.804715    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:48.805957 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:48.805927    2566 patch_prober.go:28] interesting pod/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 17 11:39:48.806101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:48.805998    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 17 11:39:49.160740 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:49.160699    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"
Jul 17 11:39:49.215427 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:49.215403    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9322e6-d2c9-46e4-a964-f1902ec2d66c-maas-api-service-ca\") pod \"3f9322e6-d2c9-46e4-a964-f1902ec2d66c\" (UID: \"3f9322e6-d2c9-46e4-a964-f1902ec2d66c\") "
Jul 17 11:39:49.215690 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:49.215670    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f9322e6-d2c9-46e4-a964-f1902ec2d66c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f9322e6-d2c9-46e4-a964-f1902ec2d66c" (UID: "3f9322e6-d2c9-46e4-a964-f1902ec2d66c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:39:49.316572 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:49.316543    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9322e6-d2c9-46e4-a964-f1902ec2d66c-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:39:49.805310 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:49.805274    2566 patch_prober.go:28] interesting pod/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 17 11:39:49.805477 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:49.805329    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 17 11:39:50.001010 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:50.000925    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85"
Jul 17 11:39:50.001010 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:50.000962    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85" event={"ID":"3f9322e6-d2c9-46e4-a964-f1902ec2d66c","Type":"ContainerDied","Data":"af0d01376d6e4f1bcbf54a722b4c8a482fc33d5f883805c654fba900c0055c03"}
Jul 17 11:39:50.001010 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:50.001008    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af0d01376d6e4f1bcbf54a722b4c8a482fc33d5f883805c654fba900c0055c03"
Jul 17 11:39:50.805222 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:50.805190    2566 patch_prober.go:28] interesting pod/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 17 11:39:50.805626 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:50.805248    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 17 11:39:51.808409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:51.808370    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:51.808832 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:51.808743    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:51.809525 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:51.809504    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:39:55.651966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:55.651931    2566 scope.go:117] "RemoveContainer" containerID="2fadb0a404a0b5f9c13ea432c90ecec34035406da47147c6165d0809500fd6ca"
Jul 17 11:39:56.033862 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:56.033836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" event={"ID":"fcdd7fde-1d32-440a-ba62-82f1410726af","Type":"ContainerStarted","Data":"c2c2454411bf3d49ac023033aeb98e9a83043cfbf184aedcfb70f957f6311487"}
Jul 17 11:39:56.051153 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:56.051102    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" podStartSLOduration=18.051085501 podStartE2EDuration="18.051085501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:39:56.050015487 +0000 UTC m=+2578.907099433" watchObservedRunningTime="2026-07-17 11:39:56.051085501 +0000 UTC m=+2578.908169436"
Jul 17 11:39:57.040783 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:57.040750    2566 generic.go:356] "Generic (PLEG): container finished" podID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerID="c2c2454411bf3d49ac023033aeb98e9a83043cfbf184aedcfb70f957f6311487" exitCode=6
Jul 17 11:39:57.041165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:57.040828    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" event={"ID":"fcdd7fde-1d32-440a-ba62-82f1410726af","Type":"ContainerDied","Data":"c2c2454411bf3d49ac023033aeb98e9a83043cfbf184aedcfb70f957f6311487"}
Jul 17 11:39:57.041165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:57.040891    2566 scope.go:117] "RemoveContainer" containerID="2fadb0a404a0b5f9c13ea432c90ecec34035406da47147c6165d0809500fd6ca"
Jul 17 11:39:57.041165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:57.041140    2566 scope.go:117] "RemoveContainer" containerID="c2c2454411bf3d49ac023033aeb98e9a83043cfbf184aedcfb70f957f6311487"
Jul 17 11:39:57.041403 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:39:57.041378    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c_odh-ai-gateway-infra(fcdd7fde-1d32-440a-ba62-82f1410726af)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af"
Jul 17 11:39:57.069546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:57.069519    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"]
Jul 17 11:39:58.187348 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:58.187324    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"
Jul 17 11:39:58.297650 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:58.297578    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdd7fde-1d32-440a-ba62-82f1410726af-maas-api-service-ca\") pod \"fcdd7fde-1d32-440a-ba62-82f1410726af\" (UID: \"fcdd7fde-1d32-440a-ba62-82f1410726af\") "
Jul 17 11:39:58.297925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:58.297904    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcdd7fde-1d32-440a-ba62-82f1410726af-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fcdd7fde-1d32-440a-ba62-82f1410726af" (UID: "fcdd7fde-1d32-440a-ba62-82f1410726af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:39:58.398628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:58.398601    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdd7fde-1d32-440a-ba62-82f1410726af-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:39:59.053315 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:59.053282    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c" event={"ID":"fcdd7fde-1d32-440a-ba62-82f1410726af","Type":"ContainerDied","Data":"1285f8da079e78411299feebce1c07f2b372e1ba2ace018c5b854aa00bd67094"}
Jul 17 11:39:59.053478 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:59.053347    2566 scope.go:117] "RemoveContainer" containerID="c2c2454411bf3d49ac023033aeb98e9a83043cfbf184aedcfb70f957f6311487"
Jul 17 11:39:59.053478 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:59.053298    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"
Jul 17 11:39:59.080028 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:59.080003    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"]
Jul 17 11:39:59.086793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:59.086773    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-rs89c"]
Jul 17 11:39:59.657717 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:59.657675    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" path="/var/lib/kubelet/pods/fcdd7fde-1d32-440a-ba62-82f1410726af/volumes"
Jul 17 11:39:59.750661 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:39:59.750633    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:40:07.438082 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438052    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"]
Jul 17 11:40:07.438504 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438423    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerName="revoke-keys"
Jul 17 11:40:07.438504 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438438    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerName="revoke-keys"
Jul 17 11:40:07.438504 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438452    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f9322e6-d2c9-46e4-a964-f1902ec2d66c" containerName="revoke-keys"
Jul 17 11:40:07.438504 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438458    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f9322e6-d2c9-46e4-a964-f1902ec2d66c" containerName="revoke-keys"
Jul 17 11:40:07.438504 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438469    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerName="revoke-keys"
Jul 17 11:40:07.438504 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438475    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerName="revoke-keys"
Jul 17 11:40:07.438695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438533    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerName="revoke-keys"
Jul 17 11:40:07.438695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438540    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerName="revoke-keys"
Jul 17 11:40:07.438695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.438547    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f9322e6-d2c9-46e4-a964-f1902ec2d66c" containerName="revoke-keys"
Jul 17 11:40:07.441480 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.441463    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"
Jul 17 11:40:07.445279 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.445249    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:40:07.445403 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.445298    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:40:07.451016 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.450989    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"]
Jul 17 11:40:07.582158 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.582130    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/877c39d9-c2b0-44d2-a1cc-da0ae233640d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-vkd67\" (UID: \"877c39d9-c2b0-44d2-a1cc-da0ae233640d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"
Jul 17 11:40:07.682973 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.682928    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/877c39d9-c2b0-44d2-a1cc-da0ae233640d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-vkd67\" (UID: \"877c39d9-c2b0-44d2-a1cc-da0ae233640d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"
Jul 17 11:40:07.683613 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.683586    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/877c39d9-c2b0-44d2-a1cc-da0ae233640d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-vkd67\" (UID: \"877c39d9-c2b0-44d2-a1cc-da0ae233640d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"
Jul 17 11:40:07.752931 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.752858    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"
Jul 17 11:40:07.883784 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:07.883760    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"]
Jul 17 11:40:07.885539 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:40:07.885498    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877c39d9_c2b0_44d2_a1cc_da0ae233640d.slice/crio-0533286fbb2e342085684366d8917f01ad586d8143410c64a4f28a8d6708965a WatchSource:0}: Error finding container 0533286fbb2e342085684366d8917f01ad586d8143410c64a4f28a8d6708965a: Status 404 returned error can't find the container with id 0533286fbb2e342085684366d8917f01ad586d8143410c64a4f28a8d6708965a
Jul 17 11:40:08.101670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:08.101635    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" event={"ID":"877c39d9-c2b0-44d2-a1cc-da0ae233640d","Type":"ContainerStarted","Data":"0533286fbb2e342085684366d8917f01ad586d8143410c64a4f28a8d6708965a"}
Jul 17 11:40:09.107327 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:09.107292    2566 generic.go:356] "Generic (PLEG): container finished" podID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerID="ce365b8518843cd1afb2caa12d0191be49b89d226a38f1c58b083ae779933c54" exitCode=6
Jul 17 11:40:09.107755 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:09.107377    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" event={"ID":"877c39d9-c2b0-44d2-a1cc-da0ae233640d","Type":"ContainerDied","Data":"ce365b8518843cd1afb2caa12d0191be49b89d226a38f1c58b083ae779933c54"}
Jul 17 11:40:09.107755 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:09.107539    2566 scope.go:117] "RemoveContainer" containerID="ce365b8518843cd1afb2caa12d0191be49b89d226a38f1c58b083ae779933c54"
Jul 17 11:40:10.114265 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:10.114225    2566 generic.go:356] "Generic (PLEG): container finished" podID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerID="2efedb864bdad2f1a4d2a43ce5197512b859a99db4440861f6600cf75af10061" exitCode=6
Jul 17 11:40:10.114694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:10.114295    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" event={"ID":"877c39d9-c2b0-44d2-a1cc-da0ae233640d","Type":"ContainerDied","Data":"2efedb864bdad2f1a4d2a43ce5197512b859a99db4440861f6600cf75af10061"}
Jul 17 11:40:10.114694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:10.114388    2566 scope.go:117] "RemoveContainer" containerID="ce365b8518843cd1afb2caa12d0191be49b89d226a38f1c58b083ae779933c54"
Jul 17 11:40:10.114694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:10.114589    2566 scope.go:117] "RemoveContainer" containerID="2efedb864bdad2f1a4d2a43ce5197512b859a99db4440861f6600cf75af10061"
Jul 17 11:40:10.114872 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:10.114819    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-vkd67_odh-ai-gateway-infra(877c39d9-c2b0-44d2-a1cc-da0ae233640d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d"
Jul 17 11:40:11.120373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:11.120346    2566 scope.go:117] "RemoveContainer" containerID="2efedb864bdad2f1a4d2a43ce5197512b859a99db4440861f6600cf75af10061"
Jul 17 11:40:11.120788 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:11.120546    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-vkd67_odh-ai-gateway-infra(877c39d9-c2b0-44d2-a1cc-da0ae233640d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d"
Jul 17 11:40:12.258498 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:12.258461    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"]
Jul 17 11:40:12.259003 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:12.258744    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy" containerID="cri-o://b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892" gracePeriod=30
Jul 17 11:40:17.514921 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.514893    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:40:17.573036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573005    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-credential-socket\") pod \"03166279-6a7f-4735-82b4-01335014fe06\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") "
Jul 17 11:40:17.573201 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573048    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlkrl\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-kube-api-access-tlkrl\") pod \"03166279-6a7f-4735-82b4-01335014fe06\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") "
Jul 17 11:40:17.573201 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573084    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03166279-6a7f-4735-82b4-01335014fe06-istiod-ca-cert\") pod \"03166279-6a7f-4735-82b4-01335014fe06\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") "
Jul 17 11:40:17.573201 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573108    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-socket\") pod \"03166279-6a7f-4735-82b4-01335014fe06\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") "
Jul 17 11:40:17.573201 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573148    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-data\") pod \"03166279-6a7f-4735-82b4-01335014fe06\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") "
Jul 17 11:40:17.573201 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573178    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-certs\") pod \"03166279-6a7f-4735-82b4-01335014fe06\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") "
Jul 17 11:40:17.573488 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573215    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-envoy\") pod \"03166279-6a7f-4735-82b4-01335014fe06\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") "
Jul 17 11:40:17.573488 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573244    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03166279-6a7f-4735-82b4-01335014fe06-istio-podinfo\") pod \"03166279-6a7f-4735-82b4-01335014fe06\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") "
Jul 17 11:40:17.573488 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573269    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03166279-6a7f-4735-82b4-01335014fe06" (UID: "03166279-6a7f-4735-82b4-01335014fe06"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:40:17.573488 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573298    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-istio-token\") pod \"03166279-6a7f-4735-82b4-01335014fe06\" (UID: \"03166279-6a7f-4735-82b4-01335014fe06\") "
Jul 17 11:40:17.573488 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573460    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03166279-6a7f-4735-82b4-01335014fe06" (UID: "03166279-6a7f-4735-82b4-01335014fe06"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:40:17.573831 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573557    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03166279-6a7f-4735-82b4-01335014fe06-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03166279-6a7f-4735-82b4-01335014fe06" (UID: "03166279-6a7f-4735-82b4-01335014fe06"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:40:17.573831 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573657    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:17.573831 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573767    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03166279-6a7f-4735-82b4-01335014fe06" (UID: "03166279-6a7f-4735-82b4-01335014fe06"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:40:17.573831 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573804    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03166279-6a7f-4735-82b4-01335014fe06-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:17.573831 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.573820    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:17.574105 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.574080    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-data" (OuterVolumeSpecName: "istio-data") pod "03166279-6a7f-4735-82b4-01335014fe06" (UID: "03166279-6a7f-4735-82b4-01335014fe06"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:40:17.575375 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.575342    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03166279-6a7f-4735-82b4-01335014fe06-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03166279-6a7f-4735-82b4-01335014fe06" (UID: "03166279-6a7f-4735-82b4-01335014fe06"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:40:17.575375 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.575368    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-istio-token" (OuterVolumeSpecName: "istio-token") pod "03166279-6a7f-4735-82b4-01335014fe06" (UID: "03166279-6a7f-4735-82b4-01335014fe06"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:40:17.575636 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.575617    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03166279-6a7f-4735-82b4-01335014fe06" (UID: "03166279-6a7f-4735-82b4-01335014fe06"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:40:17.575636 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.575627    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-kube-api-access-tlkrl" (OuterVolumeSpecName: "kube-api-access-tlkrl") pod "03166279-6a7f-4735-82b4-01335014fe06" (UID: "03166279-6a7f-4735-82b4-01335014fe06"). InnerVolumeSpecName "kube-api-access-tlkrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:40:17.675036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.675009    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlkrl\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-kube-api-access-tlkrl\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:17.675036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.675035    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:17.675178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.675045    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:17.675178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.675053    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03166279-6a7f-4735-82b4-01335014fe06-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:17.675178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.675060    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03166279-6a7f-4735-82b4-01335014fe06-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:17.675178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:17.675068    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03166279-6a7f-4735-82b4-01335014fe06-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:18.155185 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.155150    2566 generic.go:356] "Generic (PLEG): container finished" podID="03166279-6a7f-4735-82b4-01335014fe06" containerID="b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892" exitCode=0
Jul 17 11:40:18.155373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.155204    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"
Jul 17 11:40:18.155373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.155240    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" event={"ID":"03166279-6a7f-4735-82b4-01335014fe06","Type":"ContainerDied","Data":"b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892"}
Jul 17 11:40:18.155373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.155297    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd" event={"ID":"03166279-6a7f-4735-82b4-01335014fe06","Type":"ContainerDied","Data":"95e12eeffedbfb5eb5e91454e953aa26b046653f552c2604814fc6f15c521934"}
Jul 17 11:40:18.155373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.155322    2566 scope.go:117] "RemoveContainer" containerID="b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892"
Jul 17 11:40:18.163794 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.163690    2566 scope.go:117] "RemoveContainer" containerID="61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037"
Jul 17 11:40:18.171466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.171451    2566 scope.go:117] "RemoveContainer" containerID="b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892"
Jul 17 11:40:18.171739 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:18.171716    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892\": container with ID starting with b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892 not found: ID does not exist" containerID="b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892"
Jul 17 11:40:18.171809 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.171749    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892"} err="failed to get container status \"b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892\": rpc error: code = NotFound desc = could not find container \"b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892\": container with ID starting with b4f862e3a46068372745fdbf84b9451eb244154272c707a2cb9d87e66dcb3892 not found: ID does not exist"
Jul 17 11:40:18.171809 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.171766    2566 scope.go:117] "RemoveContainer" containerID="61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037"
Jul 17 11:40:18.172022 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:18.172004    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037\": container with ID starting with 61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037 not found: ID does not exist" containerID="61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037"
Jul 17 11:40:18.172071 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.172029    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037"} err="failed to get container status \"61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037\": rpc error: code = NotFound desc = could not find container \"61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037\": container with ID starting with 61f56e281cbc36f0cd8006d782a44ff8a1d7e0e35a10aa4670b084dcae773037 not found: ID does not exist"
Jul 17 11:40:18.177951 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.177918    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"]
Jul 17 11:40:18.182101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.182081    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5a6d7b05-openshift-default-b47f9d859-dqltd"]
Jul 17 11:40:18.634751 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.634700    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"]
Jul 17 11:40:18.635138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.635082    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy"
Jul 17 11:40:18.635138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.635092    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy"
Jul 17 11:40:18.635138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.635102    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerName="revoke-keys"
Jul 17 11:40:18.635138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.635107    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerName="revoke-keys"
Jul 17 11:40:18.635138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.635118    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy"
Jul 17 11:40:18.635138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.635124    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy"
Jul 17 11:40:18.635332 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.635180    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy"
Jul 17 11:40:18.635332 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.635190    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdd7fde-1d32-440a-ba62-82f1410726af" containerName="revoke-keys"
Jul 17 11:40:18.635332 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.635197    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="03166279-6a7f-4735-82b4-01335014fe06" containerName="istio-proxy"
Jul 17 11:40:18.639473 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.639454    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"
Jul 17 11:40:18.649171 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.649140    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"]
Jul 17 11:40:18.684984 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.684958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fe9804-edec-4f9d-9d8e-a71488af1477-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5\" (UID: \"c6fe9804-edec-4f9d-9d8e-a71488af1477\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"
Jul 17 11:40:18.785728 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.785676    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fe9804-edec-4f9d-9d8e-a71488af1477-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5\" (UID: \"c6fe9804-edec-4f9d-9d8e-a71488af1477\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"
Jul 17 11:40:18.786240 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.786222    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fe9804-edec-4f9d-9d8e-a71488af1477-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5\" (UID: \"c6fe9804-edec-4f9d-9d8e-a71488af1477\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"
Jul 17 11:40:18.950992 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:18.950908    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"
Jul 17 11:40:19.076577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:19.076543    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"]
Jul 17 11:40:19.079405 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:40:19.079378    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fe9804_edec_4f9d_9d8e_a71488af1477.slice/crio-1436decf7e48756a9eb8b8a8762984ebf7352f286452964ff91abd1d2e0e0fdd WatchSource:0}: Error finding container 1436decf7e48756a9eb8b8a8762984ebf7352f286452964ff91abd1d2e0e0fdd: Status 404 returned error can't find the container with id 1436decf7e48756a9eb8b8a8762984ebf7352f286452964ff91abd1d2e0e0fdd
Jul 17 11:40:19.161899 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:19.161873    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5" event={"ID":"c6fe9804-edec-4f9d-9d8e-a71488af1477","Type":"ContainerStarted","Data":"1436decf7e48756a9eb8b8a8762984ebf7352f286452964ff91abd1d2e0e0fdd"}
Jul 17 11:40:19.657721 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:19.657678    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03166279-6a7f-4735-82b4-01335014fe06" path="/var/lib/kubelet/pods/03166279-6a7f-4735-82b4-01335014fe06/volumes"
Jul 17 11:40:20.169004 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:20.168922    2566 generic.go:356] "Generic (PLEG): container finished" podID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerID="77fe614061ece2a114b5398676011b75c331fac4f43691800f1c2434b321391f" exitCode=6
Jul 17 11:40:20.169176 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:20.169004    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5" event={"ID":"c6fe9804-edec-4f9d-9d8e-a71488af1477","Type":"ContainerDied","Data":"77fe614061ece2a114b5398676011b75c331fac4f43691800f1c2434b321391f"}
Jul 17 11:40:20.169232 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:20.169186    2566 scope.go:117] "RemoveContainer" containerID="77fe614061ece2a114b5398676011b75c331fac4f43691800f1c2434b321391f"
Jul 17 11:40:21.175216 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:21.175124    2566 generic.go:356] "Generic (PLEG): container finished" podID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerID="b70b0788d7e08e29a27cbac1d6173ac37ba70f120fadc4f40a5a69791a4d8d3c" exitCode=6
Jul 17 11:40:21.175627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:21.175212    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5" event={"ID":"c6fe9804-edec-4f9d-9d8e-a71488af1477","Type":"ContainerDied","Data":"b70b0788d7e08e29a27cbac1d6173ac37ba70f120fadc4f40a5a69791a4d8d3c"}
Jul 17 11:40:21.175627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:21.175267    2566 scope.go:117] "RemoveContainer" containerID="77fe614061ece2a114b5398676011b75c331fac4f43691800f1c2434b321391f"
Jul 17 11:40:21.175627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:21.175366    2566 scope.go:117] "RemoveContainer" containerID="b70b0788d7e08e29a27cbac1d6173ac37ba70f120fadc4f40a5a69791a4d8d3c"
Jul 17 11:40:21.175627 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:21.175559    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5_odh-ai-gateway-infra(c6fe9804-edec-4f9d-9d8e-a71488af1477)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477"
Jul 17 11:40:22.182225 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:22.182197    2566 scope.go:117] "RemoveContainer" containerID="b70b0788d7e08e29a27cbac1d6173ac37ba70f120fadc4f40a5a69791a4d8d3c"
Jul 17 11:40:22.182624 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:22.182450    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5_odh-ai-gateway-infra(c6fe9804-edec-4f9d-9d8e-a71488af1477)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477"
Jul 17 11:40:23.651853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:23.651823    2566 scope.go:117] "RemoveContainer" containerID="2efedb864bdad2f1a4d2a43ce5197512b859a99db4440861f6600cf75af10061"
Jul 17 11:40:24.192724 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:24.192625    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" event={"ID":"877c39d9-c2b0-44d2-a1cc-da0ae233640d","Type":"ContainerStarted","Data":"b8f043a26fe8073e3bb2ccf78dbae0f48a1d3ff09da2ababebf75ecfc22fcd1f"}
Jul 17 11:40:24.211341 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:24.211289    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" podStartSLOduration=17.211274705 podStartE2EDuration="17.211274705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:40:24.20978608 +0000 UTC m=+2607.066870025" watchObservedRunningTime="2026-07-17 11:40:24.211274705 +0000 UTC m=+2607.068358650"
Jul 17 11:40:25.199612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:25.199579    2566 generic.go:356] "Generic (PLEG): container finished" podID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerID="b8f043a26fe8073e3bb2ccf78dbae0f48a1d3ff09da2ababebf75ecfc22fcd1f" exitCode=6
Jul 17 11:40:25.200086 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:25.199656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" event={"ID":"877c39d9-c2b0-44d2-a1cc-da0ae233640d","Type":"ContainerDied","Data":"b8f043a26fe8073e3bb2ccf78dbae0f48a1d3ff09da2ababebf75ecfc22fcd1f"}
Jul 17 11:40:25.200086 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:25.199750    2566 scope.go:117] "RemoveContainer" containerID="2efedb864bdad2f1a4d2a43ce5197512b859a99db4440861f6600cf75af10061"
Jul 17 11:40:25.200086 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:25.200060    2566 scope.go:117] "RemoveContainer" containerID="b8f043a26fe8073e3bb2ccf78dbae0f48a1d3ff09da2ababebf75ecfc22fcd1f"
Jul 17 11:40:25.200275 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:25.200253    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-vkd67_odh-ai-gateway-infra(877c39d9-c2b0-44d2-a1cc-da0ae233640d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d"
Jul 17 11:40:25.226871 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:25.226844    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"]
Jul 17 11:40:26.362549 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:26.362524    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"
Jul 17 11:40:26.449451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:26.449422    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/877c39d9-c2b0-44d2-a1cc-da0ae233640d-maas-api-service-ca\") pod \"877c39d9-c2b0-44d2-a1cc-da0ae233640d\" (UID: \"877c39d9-c2b0-44d2-a1cc-da0ae233640d\") "
Jul 17 11:40:26.449786 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:26.449764    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/877c39d9-c2b0-44d2-a1cc-da0ae233640d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "877c39d9-c2b0-44d2-a1cc-da0ae233640d" (UID: "877c39d9-c2b0-44d2-a1cc-da0ae233640d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:40:26.550469 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:26.550375    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/877c39d9-c2b0-44d2-a1cc-da0ae233640d-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:27.217481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:27.217453    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"
Jul 17 11:40:27.217639 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:27.217449    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67" event={"ID":"877c39d9-c2b0-44d2-a1cc-da0ae233640d","Type":"ContainerDied","Data":"0533286fbb2e342085684366d8917f01ad586d8143410c64a4f28a8d6708965a"}
Jul 17 11:40:27.217639 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:27.217609    2566 scope.go:117] "RemoveContainer" containerID="b8f043a26fe8073e3bb2ccf78dbae0f48a1d3ff09da2ababebf75ecfc22fcd1f"
Jul 17 11:40:27.243358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:27.243331    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"]
Jul 17 11:40:27.249121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:27.249100    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vkd67"]
Jul 17 11:40:27.657989 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:27.657959    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" path="/var/lib/kubelet/pods/877c39d9-c2b0-44d2-a1cc-da0ae233640d/volumes"
Jul 17 11:40:32.651841 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:32.651812    2566 scope.go:117] "RemoveContainer" containerID="b70b0788d7e08e29a27cbac1d6173ac37ba70f120fadc4f40a5a69791a4d8d3c"
Jul 17 11:40:33.249579 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.249552    2566 generic.go:356] "Generic (PLEG): container finished" podID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerID="e1f42fe725fa53826b035450814364892f6a00c3c11cfa51791b2cd7bd658e82" exitCode=6
Jul 17 11:40:33.249688 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.249622    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5" event={"ID":"c6fe9804-edec-4f9d-9d8e-a71488af1477","Type":"ContainerDied","Data":"e1f42fe725fa53826b035450814364892f6a00c3c11cfa51791b2cd7bd658e82"}
Jul 17 11:40:33.249688 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.249680    2566 scope.go:117] "RemoveContainer" containerID="b70b0788d7e08e29a27cbac1d6173ac37ba70f120fadc4f40a5a69791a4d8d3c"
Jul 17 11:40:33.249987 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.249974    2566 scope.go:117] "RemoveContainer" containerID="e1f42fe725fa53826b035450814364892f6a00c3c11cfa51791b2cd7bd658e82"
Jul 17 11:40:33.256025 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:33.250784    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5_odh-ai-gateway-infra(c6fe9804-edec-4f9d-9d8e-a71488af1477)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477"
Jul 17 11:40:33.397570 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.397536    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"]
Jul 17 11:40:33.398194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.398159    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerName="revoke-keys"
Jul 17 11:40:33.398194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.398182    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerName="revoke-keys"
Jul 17 11:40:33.398391 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.398202    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerName="revoke-keys"
Jul 17 11:40:33.398391 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.398211    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerName="revoke-keys"
Jul 17 11:40:33.398391 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.398290    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerName="revoke-keys"
Jul 17 11:40:33.398391 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.398303    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerName="revoke-keys"
Jul 17 11:40:33.402985 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.402967    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.406963 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.406944    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-63359af2-openshift-default-dockercfg-tbm84\""
Jul 17 11:40:33.415157 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.415130    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"]
Jul 17 11:40:33.508084 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.508016    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-podinfo\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.508084 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.508065    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c74c80eb-4ad6-431c-b227-2669d494ec2b-istiod-ca-cert\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.508300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.508085    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql5wv\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-kube-api-access-ql5wv\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.508300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.508179    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-credential-socket\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.508300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.508227    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-socket\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.508300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.508253    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-token\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.508300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.508286    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-envoy\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.508477 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.508337    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-certs\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.508477 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.508359    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-data\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609436 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609401    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-data\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609445    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-podinfo\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c74c80eb-4ad6-431c-b227-2669d494ec2b-istiod-ca-cert\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609512    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5wv\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-kube-api-access-ql5wv\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609549    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-credential-socket\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-socket\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609613    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-token\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609645    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-envoy\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609678    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-certs\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.609887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609836    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-data\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.610066 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.609902    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-credential-socket\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.610066 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.610036    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-certs\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.610157 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.610138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-socket\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.610313 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.610294    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c74c80eb-4ad6-431c-b227-2669d494ec2b-istiod-ca-cert\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.611860 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.611842    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-envoy\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.612007 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.611993    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-podinfo\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.618867 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.618828    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-token\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.619242 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.619219    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql5wv\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-kube-api-access-ql5wv\") pod \"e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") " pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.712808 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.712774    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:33.854365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.854331    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"]
Jul 17 11:40:33.857239 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:40:33.857212    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc74c80eb_4ad6_431c_b227_2669d494ec2b.slice/crio-78d343a93ff1d12a1a41f47eef60aaa4971f8062069566fd87f4bde492f10498 WatchSource:0}: Error finding container 78d343a93ff1d12a1a41f47eef60aaa4971f8062069566fd87f4bde492f10498: Status 404 returned error can't find the container with id 78d343a93ff1d12a1a41f47eef60aaa4971f8062069566fd87f4bde492f10498
Jul 17 11:40:33.859461 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.859424    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:40:33.859573 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.859508    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:40:33.859573 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:33.859551    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:40:34.257471 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.257434    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" event={"ID":"c74c80eb-4ad6-431c-b227-2669d494ec2b","Type":"ContainerStarted","Data":"736a7f14639bfe4a53e8cb2f21a4206397115a5d98f5c34c0d2705f6062f7c72"}
Jul 17 11:40:34.257660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.257491    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" event={"ID":"c74c80eb-4ad6-431c-b227-2669d494ec2b","Type":"ContainerStarted","Data":"78d343a93ff1d12a1a41f47eef60aaa4971f8062069566fd87f4bde492f10498"}
Jul 17 11:40:34.293332 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.293279    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" podStartSLOduration=1.293263347 podStartE2EDuration="1.293263347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:40:34.277767245 +0000 UTC m=+2617.134851211" watchObservedRunningTime="2026-07-17 11:40:34.293263347 +0000 UTC m=+2617.150347270"
Jul 17 11:40:34.293835 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.293816    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"]
Jul 17 11:40:34.440403 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.440381    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"
Jul 17 11:40:34.520033 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.519948    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fe9804-edec-4f9d-9d8e-a71488af1477-maas-api-service-ca\") pod \"c6fe9804-edec-4f9d-9d8e-a71488af1477\" (UID: \"c6fe9804-edec-4f9d-9d8e-a71488af1477\") "
Jul 17 11:40:34.520293 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.520271    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6fe9804-edec-4f9d-9d8e-a71488af1477-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c6fe9804-edec-4f9d-9d8e-a71488af1477" (UID: "c6fe9804-edec-4f9d-9d8e-a71488af1477"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:40:34.621378 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.621346    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fe9804-edec-4f9d-9d8e-a71488af1477-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:34.713668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.713634    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:34.718238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:34.718216    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:35.264166 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:35.264136    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"
Jul 17 11:40:35.264349 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:35.264167    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5" event={"ID":"c6fe9804-edec-4f9d-9d8e-a71488af1477","Type":"ContainerDied","Data":"1436decf7e48756a9eb8b8a8762984ebf7352f286452964ff91abd1d2e0e0fdd"}
Jul 17 11:40:35.264349 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:35.264223    2566 scope.go:117] "RemoveContainer" containerID="e1f42fe725fa53826b035450814364892f6a00c3c11cfa51791b2cd7bd658e82"
Jul 17 11:40:35.264652 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:35.264627    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:35.265839 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:35.265822    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:35.308817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:35.308786    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"]
Jul 17 11:40:35.317794 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:35.317767    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-xzhr5"]
Jul 17 11:40:35.658255 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:35.658222    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" path="/var/lib/kubelet/pods/c6fe9804-edec-4f9d-9d8e-a71488af1477/volumes"
Jul 17 11:40:38.592270 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592234    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"]
Jul 17 11:40:38.592673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592618    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerName="revoke-keys"
Jul 17 11:40:38.592673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592630    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerName="revoke-keys"
Jul 17 11:40:38.592673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592650    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerName="revoke-keys"
Jul 17 11:40:38.592673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592657    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerName="revoke-keys"
Jul 17 11:40:38.592673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592675    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerName="revoke-keys"
Jul 17 11:40:38.592872 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592683    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerName="revoke-keys"
Jul 17 11:40:38.592872 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592690    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerName="revoke-keys"
Jul 17 11:40:38.592872 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592696    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerName="revoke-keys"
Jul 17 11:40:38.592872 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592769    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerName="revoke-keys"
Jul 17 11:40:38.592872 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592778    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerName="revoke-keys"
Jul 17 11:40:38.592872 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.592784    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="877c39d9-c2b0-44d2-a1cc-da0ae233640d" containerName="revoke-keys"
Jul 17 11:40:38.595637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.595621    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"
Jul 17 11:40:38.598261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.598240    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:40:38.598383 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.598246    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:40:38.604147 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.604126    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"]
Jul 17 11:40:38.659750 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.659691    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73079714-8c80-4a4d-9363-a26adf7a75a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp\" (UID: \"73079714-8c80-4a4d-9363-a26adf7a75a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"
Jul 17 11:40:38.760586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.760539    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73079714-8c80-4a4d-9363-a26adf7a75a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp\" (UID: \"73079714-8c80-4a4d-9363-a26adf7a75a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"
Jul 17 11:40:38.761210 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.761189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73079714-8c80-4a4d-9363-a26adf7a75a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp\" (UID: \"73079714-8c80-4a4d-9363-a26adf7a75a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"
Jul 17 11:40:38.906651 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:38.906560    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"
Jul 17 11:40:39.038058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:39.038027    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"]
Jul 17 11:40:39.039417 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:40:39.039396    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73079714_8c80_4a4d_9363_a26adf7a75a7.slice/crio-71be08675ba360684881680998c1af7b197c45712151db6052d7797515fe8bb3 WatchSource:0}: Error finding container 71be08675ba360684881680998c1af7b197c45712151db6052d7797515fe8bb3: Status 404 returned error can't find the container with id 71be08675ba360684881680998c1af7b197c45712151db6052d7797515fe8bb3
Jul 17 11:40:39.286206 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:39.286175    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp" event={"ID":"73079714-8c80-4a4d-9363-a26adf7a75a7","Type":"ContainerStarted","Data":"71be08675ba360684881680998c1af7b197c45712151db6052d7797515fe8bb3"}
Jul 17 11:40:40.292840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:40.292806    2566 generic.go:356] "Generic (PLEG): container finished" podID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerID="3fd63e01e20babfc168b3649cf850d43e0df5e8ab28636fc6c6e8c768a868089" exitCode=6
Jul 17 11:40:40.293236 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:40.292889    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp" event={"ID":"73079714-8c80-4a4d-9363-a26adf7a75a7","Type":"ContainerDied","Data":"3fd63e01e20babfc168b3649cf850d43e0df5e8ab28636fc6c6e8c768a868089"}
Jul 17 11:40:40.293236 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:40.293064    2566 scope.go:117] "RemoveContainer" containerID="3fd63e01e20babfc168b3649cf850d43e0df5e8ab28636fc6c6e8c768a868089"
Jul 17 11:40:41.145317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:41.145277    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:40:41.300106 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:41.300013    2566 generic.go:356] "Generic (PLEG): container finished" podID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerID="da4568bd772baf60d0d846578f49ec07d682acc85814882db2f9239707b63817" exitCode=6
Jul 17 11:40:41.300106 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:41.300055    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp" event={"ID":"73079714-8c80-4a4d-9363-a26adf7a75a7","Type":"ContainerDied","Data":"da4568bd772baf60d0d846578f49ec07d682acc85814882db2f9239707b63817"}
Jul 17 11:40:41.300605 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:41.300148    2566 scope.go:117] "RemoveContainer" containerID="3fd63e01e20babfc168b3649cf850d43e0df5e8ab28636fc6c6e8c768a868089"
Jul 17 11:40:41.300605 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:41.300371    2566 scope.go:117] "RemoveContainer" containerID="da4568bd772baf60d0d846578f49ec07d682acc85814882db2f9239707b63817"
Jul 17 11:40:41.300745 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:41.300699    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp_odh-ai-gateway-infra(73079714-8c80-4a4d-9363-a26adf7a75a7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7"
Jul 17 11:40:42.306368 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:42.306340    2566 scope.go:117] "RemoveContainer" containerID="da4568bd772baf60d0d846578f49ec07d682acc85814882db2f9239707b63817"
Jul 17 11:40:42.306782 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:42.306608    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp_odh-ai-gateway-infra(73079714-8c80-4a4d-9363-a26adf7a75a7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7"
Jul 17 11:40:45.362793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:45.362757    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"]
Jul 17 11:40:45.363194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:45.363185    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6fe9804-edec-4f9d-9d8e-a71488af1477" containerName="revoke-keys"
Jul 17 11:40:45.367344 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:45.367325    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"
Jul 17 11:40:45.373981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:45.373956    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"]
Jul 17 11:40:45.521576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:45.521539    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e9953-e6b1-44c4-a628-b59a9d59df20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-t5wkz\" (UID: \"ff3e9953-e6b1-44c4-a628-b59a9d59df20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"
Jul 17 11:40:45.623144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:45.623059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e9953-e6b1-44c4-a628-b59a9d59df20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-t5wkz\" (UID: \"ff3e9953-e6b1-44c4-a628-b59a9d59df20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"
Jul 17 11:40:45.623675 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:45.623655    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e9953-e6b1-44c4-a628-b59a9d59df20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-t5wkz\" (UID: \"ff3e9953-e6b1-44c4-a628-b59a9d59df20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"
Jul 17 11:40:45.679894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:45.679866    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"
Jul 17 11:40:45.856658 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:45.856547    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"]
Jul 17 11:40:45.859146 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:40:45.859118    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3e9953_e6b1_44c4_a628_b59a9d59df20.slice/crio-0583414488c5b2d7b710c6db4c9836e23e3745d31f93ef58911d39aea0318701 WatchSource:0}: Error finding container 0583414488c5b2d7b710c6db4c9836e23e3745d31f93ef58911d39aea0318701: Status 404 returned error can't find the container with id 0583414488c5b2d7b710c6db4c9836e23e3745d31f93ef58911d39aea0318701
Jul 17 11:40:46.331176 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:46.331130    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" event={"ID":"ff3e9953-e6b1-44c4-a628-b59a9d59df20","Type":"ContainerStarted","Data":"bc2070666e08704668900f951ad3ca73c7055f38ee63fc19fca74a782dabc343"}
Jul 17 11:40:46.331342 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:46.331205    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" event={"ID":"ff3e9953-e6b1-44c4-a628-b59a9d59df20","Type":"ContainerStarted","Data":"0583414488c5b2d7b710c6db4c9836e23e3745d31f93ef58911d39aea0318701"}
Jul 17 11:40:46.362876 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:46.362830    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" podStartSLOduration=1.362815534 podStartE2EDuration="1.362815534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:40:46.359168572 +0000 UTC m=+2629.216252517" watchObservedRunningTime="2026-07-17 11:40:46.362815534 +0000 UTC m=+2629.219899479"
Jul 17 11:40:47.338491 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:47.338455    2566 generic.go:356] "Generic (PLEG): container finished" podID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerID="bc2070666e08704668900f951ad3ca73c7055f38ee63fc19fca74a782dabc343" exitCode=6
Jul 17 11:40:47.338698 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:47.338542    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" event={"ID":"ff3e9953-e6b1-44c4-a628-b59a9d59df20","Type":"ContainerDied","Data":"bc2070666e08704668900f951ad3ca73c7055f38ee63fc19fca74a782dabc343"}
Jul 17 11:40:47.338819 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:47.338802    2566 scope.go:117] "RemoveContainer" containerID="bc2070666e08704668900f951ad3ca73c7055f38ee63fc19fca74a782dabc343"
Jul 17 11:40:48.349372 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:48.349337    2566 generic.go:356] "Generic (PLEG): container finished" podID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerID="b1cc8de38ea678543d3bd5c00fba0b3850059a9ed1fbf962ad7e4c72d0f73cd4" exitCode=6
Jul 17 11:40:48.349870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:48.349410    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" event={"ID":"ff3e9953-e6b1-44c4-a628-b59a9d59df20","Type":"ContainerDied","Data":"b1cc8de38ea678543d3bd5c00fba0b3850059a9ed1fbf962ad7e4c72d0f73cd4"}
Jul 17 11:40:48.349870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:48.349496    2566 scope.go:117] "RemoveContainer" containerID="bc2070666e08704668900f951ad3ca73c7055f38ee63fc19fca74a782dabc343"
Jul 17 11:40:48.349870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:48.349744    2566 scope.go:117] "RemoveContainer" containerID="b1cc8de38ea678543d3bd5c00fba0b3850059a9ed1fbf962ad7e4c72d0f73cd4"
Jul 17 11:40:48.350047 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:48.350005    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-t5wkz_odh-ai-gateway-infra(ff3e9953-e6b1-44c4-a628-b59a9d59df20)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20"
Jul 17 11:40:49.356304 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:49.356276    2566 scope.go:117] "RemoveContainer" containerID="b1cc8de38ea678543d3bd5c00fba0b3850059a9ed1fbf962ad7e4c72d0f73cd4"
Jul 17 11:40:49.356826 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:49.356502    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-t5wkz_odh-ai-gateway-infra(ff3e9953-e6b1-44c4-a628-b59a9d59df20)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20"
Jul 17 11:40:49.357721 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:49.357681    2566 generic.go:356] "Generic (PLEG): container finished" podID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerID="736a7f14639bfe4a53e8cb2f21a4206397115a5d98f5c34c0d2705f6062f7c72" exitCode=0
Jul 17 11:40:49.357843 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:49.357735    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" event={"ID":"c74c80eb-4ad6-431c-b227-2669d494ec2b","Type":"ContainerDied","Data":"736a7f14639bfe4a53e8cb2f21a4206397115a5d98f5c34c0d2705f6062f7c72"}
Jul 17 11:40:49.358104 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:49.358088    2566 scope.go:117] "RemoveContainer" containerID="736a7f14639bfe4a53e8cb2f21a4206397115a5d98f5c34c0d2705f6062f7c72"
Jul 17 11:40:49.359051 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:49.359025    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:40:49.359113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:49.359088    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:40:49.359151 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:49.359120    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:40:50.264718 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:50.264664    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:50.364619 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:50.364582    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" event={"ID":"c74c80eb-4ad6-431c-b227-2669d494ec2b","Type":"ContainerStarted","Data":"3cfbe0824af2c7042ee8bf97067b6168f2324f0096bc2d75da8627e659a3584c"}
Jul 17 11:40:50.713239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:50.713197    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:50.714515 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:50.714488    2566 patch_prober.go:28] interesting pod/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Jul 17 11:40:50.714670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:50.714558    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Jul 17 11:40:51.713960 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:51.713922    2566 patch_prober.go:28] interesting pod/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Jul 17 11:40:51.714361 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:51.713985    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Jul 17 11:40:52.714163 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:52.714123    2566 patch_prober.go:28] interesting pod/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Jul 17 11:40:52.714575 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:52.714181    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Jul 17 11:40:53.714056 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:53.714023    2566 patch_prober.go:28] interesting pod/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Jul 17 11:40:53.714262 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:53.714082    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Jul 17 11:40:54.718307 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:54.718273    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:54.718693 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:54.718524    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:54.719170 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:54.719150    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:40:56.651803 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:56.651771    2566 scope.go:117] "RemoveContainer" containerID="da4568bd772baf60d0d846578f49ec07d682acc85814882db2f9239707b63817"
Jul 17 11:40:57.408067 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:57.408042    2566 generic.go:356] "Generic (PLEG): container finished" podID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerID="01d6853df0e965adb4c00dac80e73bea9498530d9a50e0ffab6bbbd484294f43" exitCode=6
Jul 17 11:40:57.408206 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:57.408114    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp" event={"ID":"73079714-8c80-4a4d-9363-a26adf7a75a7","Type":"ContainerDied","Data":"01d6853df0e965adb4c00dac80e73bea9498530d9a50e0ffab6bbbd484294f43"}
Jul 17 11:40:57.408206 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:57.408172    2566 scope.go:117] "RemoveContainer" containerID="da4568bd772baf60d0d846578f49ec07d682acc85814882db2f9239707b63817"
Jul 17 11:40:57.408469 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:57.408449    2566 scope.go:117] "RemoveContainer" containerID="01d6853df0e965adb4c00dac80e73bea9498530d9a50e0ffab6bbbd484294f43"
Jul 17 11:40:57.408678 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:40:57.408650    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp_odh-ai-gateway-infra(73079714-8c80-4a4d-9363-a26adf7a75a7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7"
Jul 17 11:40:57.645397 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:57.645320    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:40:58.438920 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:58.438882    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"]
Jul 17 11:40:58.562987 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:58.562966    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"
Jul 17 11:40:58.744891 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:58.744810    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73079714-8c80-4a4d-9363-a26adf7a75a7-maas-api-service-ca\") pod \"73079714-8c80-4a4d-9363-a26adf7a75a7\" (UID: \"73079714-8c80-4a4d-9363-a26adf7a75a7\") "
Jul 17 11:40:58.745160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:58.745126    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73079714-8c80-4a4d-9363-a26adf7a75a7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73079714-8c80-4a4d-9363-a26adf7a75a7" (UID: "73079714-8c80-4a4d-9363-a26adf7a75a7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:40:58.745316 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:58.745288    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73079714-8c80-4a4d-9363-a26adf7a75a7-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:40:59.421776 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:59.421751    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"
Jul 17 11:40:59.421959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:59.421745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp" event={"ID":"73079714-8c80-4a4d-9363-a26adf7a75a7","Type":"ContainerDied","Data":"71be08675ba360684881680998c1af7b197c45712151db6052d7797515fe8bb3"}
Jul 17 11:40:59.421959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:59.421889    2566 scope.go:117] "RemoveContainer" containerID="01d6853df0e965adb4c00dac80e73bea9498530d9a50e0ffab6bbbd484294f43"
Jul 17 11:40:59.445453 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:59.445428    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"]
Jul 17 11:40:59.451842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:59.451820    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vqhdp"]
Jul 17 11:40:59.657199 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:40:59.657165    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" path="/var/lib/kubelet/pods/73079714-8c80-4a4d-9363-a26adf7a75a7/volumes"
Jul 17 11:41:04.651541 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:04.651512    2566 scope.go:117] "RemoveContainer" containerID="b1cc8de38ea678543d3bd5c00fba0b3850059a9ed1fbf962ad7e4c72d0f73cd4"
Jul 17 11:41:05.454764 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:05.454732    2566 generic.go:356] "Generic (PLEG): container finished" podID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerID="11024bb404b44fec87406aa13d4430b970e0aa61cbcc2cff2365e0bf4e069e17" exitCode=6
Jul 17 11:41:05.454939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:05.454811    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" event={"ID":"ff3e9953-e6b1-44c4-a628-b59a9d59df20","Type":"ContainerDied","Data":"11024bb404b44fec87406aa13d4430b970e0aa61cbcc2cff2365e0bf4e069e17"}
Jul 17 11:41:05.454939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:05.454872    2566 scope.go:117] "RemoveContainer" containerID="b1cc8de38ea678543d3bd5c00fba0b3850059a9ed1fbf962ad7e4c72d0f73cd4"
Jul 17 11:41:05.455197 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:05.455178    2566 scope.go:117] "RemoveContainer" containerID="11024bb404b44fec87406aa13d4430b970e0aa61cbcc2cff2365e0bf4e069e17"
Jul 17 11:41:05.455417 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:05.455396    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-t5wkz_odh-ai-gateway-infra(ff3e9953-e6b1-44c4-a628-b59a9d59df20)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20"
Jul 17 11:41:06.488218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:06.488184    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"]
Jul 17 11:41:06.620165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:06.620144    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"
Jul 17 11:41:06.714983 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:06.714957    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e9953-e6b1-44c4-a628-b59a9d59df20-maas-api-service-ca\") pod \"ff3e9953-e6b1-44c4-a628-b59a9d59df20\" (UID: \"ff3e9953-e6b1-44c4-a628-b59a9d59df20\") "
Jul 17 11:41:06.715283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:06.715262    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3e9953-e6b1-44c4-a628-b59a9d59df20-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff3e9953-e6b1-44c4-a628-b59a9d59df20" (UID: "ff3e9953-e6b1-44c4-a628-b59a9d59df20"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:41:06.816002 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:06.815944    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e9953-e6b1-44c4-a628-b59a9d59df20-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:41:07.468303 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:07.468271    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"
Jul 17 11:41:07.468481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:07.468308    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz" event={"ID":"ff3e9953-e6b1-44c4-a628-b59a9d59df20","Type":"ContainerDied","Data":"0583414488c5b2d7b710c6db4c9836e23e3745d31f93ef58911d39aea0318701"}
Jul 17 11:41:07.468481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:07.468371    2566 scope.go:117] "RemoveContainer" containerID="11024bb404b44fec87406aa13d4430b970e0aa61cbcc2cff2365e0bf4e069e17"
Jul 17 11:41:07.491405 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:07.491375    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"]
Jul 17 11:41:07.493354 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:07.493332    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-t5wkz"]
Jul 17 11:41:07.657579 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:07.657549    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" path="/var/lib/kubelet/pods/ff3e9953-e6b1-44c4-a628-b59a9d59df20/volumes"
Jul 17 11:41:08.694369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694332    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"]
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694676    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerName="revoke-keys"
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694687    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerName="revoke-keys"
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694698    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerName="revoke-keys"
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694717    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerName="revoke-keys"
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694732    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerName="revoke-keys"
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694737    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerName="revoke-keys"
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694746    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerName="revoke-keys"
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694751    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerName="revoke-keys"
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694759    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerName="revoke-keys"
Jul 17 11:41:08.694780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694764    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerName="revoke-keys"
Jul 17 11:41:08.695099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694816    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerName="revoke-keys"
Jul 17 11:41:08.695099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694823    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerName="revoke-keys"
Jul 17 11:41:08.695099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694831    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerName="revoke-keys"
Jul 17 11:41:08.695099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694836    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerName="revoke-keys"
Jul 17 11:41:08.695099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.694844    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerName="revoke-keys"
Jul 17 11:41:08.699081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.699062    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"
Jul 17 11:41:08.701931 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.701912    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:41:08.702034 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.701912    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:41:08.712912 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.712885    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"]
Jul 17 11:41:08.731132 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.731107    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73672302-ceea-4e40-9bac-ccd8db50d46f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-s87zd\" (UID: \"73672302-ceea-4e40-9bac-ccd8db50d46f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"
Jul 17 11:41:08.832237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.832192    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73672302-ceea-4e40-9bac-ccd8db50d46f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-s87zd\" (UID: \"73672302-ceea-4e40-9bac-ccd8db50d46f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"
Jul 17 11:41:08.832826 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:08.832808    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73672302-ceea-4e40-9bac-ccd8db50d46f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-s87zd\" (UID: \"73672302-ceea-4e40-9bac-ccd8db50d46f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"
Jul 17 11:41:09.014555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:09.014470    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"
Jul 17 11:41:09.141284 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:09.141246    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"]
Jul 17 11:41:09.143003 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:41:09.142963    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73672302_ceea_4e40_9bac_ccd8db50d46f.slice/crio-22164c1e9e1bc8fdf54026a5f9bba9e846a049021415739aec70e343a7ba34c5 WatchSource:0}: Error finding container 22164c1e9e1bc8fdf54026a5f9bba9e846a049021415739aec70e343a7ba34c5: Status 404 returned error can't find the container with id 22164c1e9e1bc8fdf54026a5f9bba9e846a049021415739aec70e343a7ba34c5
Jul 17 11:41:09.479776 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:09.479741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" event={"ID":"73672302-ceea-4e40-9bac-ccd8db50d46f","Type":"ContainerStarted","Data":"fb2f96e6bb98d2cdbed85cc3203bdb3e3cd20313232632e9540b01815d2a7df5"}
Jul 17 11:41:09.479891 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:09.479794    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" event={"ID":"73672302-ceea-4e40-9bac-ccd8db50d46f","Type":"ContainerStarted","Data":"22164c1e9e1bc8fdf54026a5f9bba9e846a049021415739aec70e343a7ba34c5"}
Jul 17 11:41:09.497925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:09.497881    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" podStartSLOduration=1.497868185 podStartE2EDuration="1.497868185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:41:09.496432072 +0000 UTC m=+2652.353516018" watchObservedRunningTime="2026-07-17 11:41:09.497868185 +0000 UTC m=+2652.354952156"
Jul 17 11:41:10.485807 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:10.485775    2566 generic.go:356] "Generic (PLEG): container finished" podID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerID="fb2f96e6bb98d2cdbed85cc3203bdb3e3cd20313232632e9540b01815d2a7df5" exitCode=6
Jul 17 11:41:10.486219 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:10.485872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" event={"ID":"73672302-ceea-4e40-9bac-ccd8db50d46f","Type":"ContainerDied","Data":"fb2f96e6bb98d2cdbed85cc3203bdb3e3cd20313232632e9540b01815d2a7df5"}
Jul 17 11:41:10.486219 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:10.486052    2566 scope.go:117] "RemoveContainer" containerID="fb2f96e6bb98d2cdbed85cc3203bdb3e3cd20313232632e9540b01815d2a7df5"
Jul 17 11:41:11.492338 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:11.492296    2566 generic.go:356] "Generic (PLEG): container finished" podID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerID="f9f2c47886265c368bd5a284c2669c579a871b6320c06a50d0e59234fa645802" exitCode=6
Jul 17 11:41:11.492796 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:11.492373    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" event={"ID":"73672302-ceea-4e40-9bac-ccd8db50d46f","Type":"ContainerDied","Data":"f9f2c47886265c368bd5a284c2669c579a871b6320c06a50d0e59234fa645802"}
Jul 17 11:41:11.492796 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:11.492432    2566 scope.go:117] "RemoveContainer" containerID="fb2f96e6bb98d2cdbed85cc3203bdb3e3cd20313232632e9540b01815d2a7df5"
Jul 17 11:41:11.492796 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:11.492596    2566 scope.go:117] "RemoveContainer" containerID="f9f2c47886265c368bd5a284c2669c579a871b6320c06a50d0e59234fa645802"
Jul 17 11:41:11.492912 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:11.492862    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-s87zd_odh-ai-gateway-infra(73672302-ceea-4e40-9bac-ccd8db50d46f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f"
Jul 17 11:41:12.499349 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:12.499322    2566 scope.go:117] "RemoveContainer" containerID="f9f2c47886265c368bd5a284c2669c579a871b6320c06a50d0e59234fa645802"
Jul 17 11:41:12.499727 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:12.499525    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-s87zd_odh-ai-gateway-infra(73672302-ceea-4e40-9bac-ccd8db50d46f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f"
Jul 17 11:41:12.751409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:12.751327    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:41:23.652273 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:23.652240    2566 scope.go:117] "RemoveContainer" containerID="f9f2c47886265c368bd5a284c2669c579a871b6320c06a50d0e59234fa645802"
Jul 17 11:41:24.560483 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:24.560392    2566 generic.go:356] "Generic (PLEG): container finished" podID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerID="f0020868e963be6fd4d0bca99b3c8f4d19695bf8b5f1da2581a77bb694919fba" exitCode=6
Jul 17 11:41:24.560668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:24.560469    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" event={"ID":"73672302-ceea-4e40-9bac-ccd8db50d46f","Type":"ContainerDied","Data":"f0020868e963be6fd4d0bca99b3c8f4d19695bf8b5f1da2581a77bb694919fba"}
Jul 17 11:41:24.560668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:24.560545    2566 scope.go:117] "RemoveContainer" containerID="f9f2c47886265c368bd5a284c2669c579a871b6320c06a50d0e59234fa645802"
Jul 17 11:41:24.560839 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:24.560824    2566 scope.go:117] "RemoveContainer" containerID="f0020868e963be6fd4d0bca99b3c8f4d19695bf8b5f1da2581a77bb694919fba"
Jul 17 11:41:24.561037 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:24.561016    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-s87zd_odh-ai-gateway-infra(73672302-ceea-4e40-9bac-ccd8db50d46f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f"
Jul 17 11:41:25.590601 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:25.590565    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"]
Jul 17 11:41:25.729561 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:25.729537    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"
Jul 17 11:41:25.790291 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:25.790259    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73672302-ceea-4e40-9bac-ccd8db50d46f-maas-api-service-ca\") pod \"73672302-ceea-4e40-9bac-ccd8db50d46f\" (UID: \"73672302-ceea-4e40-9bac-ccd8db50d46f\") "
Jul 17 11:41:25.790896 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:25.790595    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73672302-ceea-4e40-9bac-ccd8db50d46f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73672302-ceea-4e40-9bac-ccd8db50d46f" (UID: "73672302-ceea-4e40-9bac-ccd8db50d46f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:41:25.790896 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:25.790867    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73672302-ceea-4e40-9bac-ccd8db50d46f-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:41:26.572565 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:26.572536    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"
Jul 17 11:41:26.572565 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:26.572557    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd" event={"ID":"73672302-ceea-4e40-9bac-ccd8db50d46f","Type":"ContainerDied","Data":"22164c1e9e1bc8fdf54026a5f9bba9e846a049021415739aec70e343a7ba34c5"}
Jul 17 11:41:26.572837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:26.572618    2566 scope.go:117] "RemoveContainer" containerID="f0020868e963be6fd4d0bca99b3c8f4d19695bf8b5f1da2581a77bb694919fba"
Jul 17 11:41:26.595981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:26.595954    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"]
Jul 17 11:41:26.603832 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:26.603811    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s87zd"]
Jul 17 11:41:27.659128 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:27.659090    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" path="/var/lib/kubelet/pods/73672302-ceea-4e40-9bac-ccd8db50d46f/volumes"
Jul 17 11:41:33.835362 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:33.835317    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:41:38.645701 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.645666    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"]
Jul 17 11:41:38.646204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.646023    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerName="revoke-keys"
Jul 17 11:41:38.646204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.646034    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="73079714-8c80-4a4d-9363-a26adf7a75a7" containerName="revoke-keys"
Jul 17 11:41:38.646204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.646046    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerName="revoke-keys"
Jul 17 11:41:38.646204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.646052    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerName="revoke-keys"
Jul 17 11:41:38.646204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.646068    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerName="revoke-keys"
Jul 17 11:41:38.646204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.646074    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerName="revoke-keys"
Jul 17 11:41:38.646204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.646122    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerName="revoke-keys"
Jul 17 11:41:38.646204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.646131    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerName="revoke-keys"
Jul 17 11:41:38.646204 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.646139    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff3e9953-e6b1-44c4-a628-b59a9d59df20" containerName="revoke-keys"
Jul 17 11:41:38.649127 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.649112    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"
Jul 17 11:41:38.652290 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.652261    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:41:38.652290 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.652268    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:41:38.659938 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.659914    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"]
Jul 17 11:41:38.810040 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.809996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97882150-f170-4d9f-95dc-b2c90b79760f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz\" (UID: \"97882150-f170-4d9f-95dc-b2c90b79760f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"
Jul 17 11:41:38.911161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.911088    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97882150-f170-4d9f-95dc-b2c90b79760f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz\" (UID: \"97882150-f170-4d9f-95dc-b2c90b79760f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"
Jul 17 11:41:38.911692 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.911671    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97882150-f170-4d9f-95dc-b2c90b79760f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz\" (UID: \"97882150-f170-4d9f-95dc-b2c90b79760f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"
Jul 17 11:41:38.960276 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:38.960251    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"
Jul 17 11:41:39.088102 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:39.088073    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"]
Jul 17 11:41:39.090083 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:41:39.090050    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97882150_f170_4d9f_95dc_b2c90b79760f.slice/crio-112b85a361ce4b7ad79e38fabc0c393c5a8eca0bd0260b32bbd7dde943155974 WatchSource:0}: Error finding container 112b85a361ce4b7ad79e38fabc0c393c5a8eca0bd0260b32bbd7dde943155974: Status 404 returned error can't find the container with id 112b85a361ce4b7ad79e38fabc0c393c5a8eca0bd0260b32bbd7dde943155974
Jul 17 11:41:39.642631 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:39.642607    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz" event={"ID":"97882150-f170-4d9f-95dc-b2c90b79760f","Type":"ContainerStarted","Data":"fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa"}
Jul 17 11:41:39.642773 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:39.642653    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz" event={"ID":"97882150-f170-4d9f-95dc-b2c90b79760f","Type":"ContainerStarted","Data":"112b85a361ce4b7ad79e38fabc0c393c5a8eca0bd0260b32bbd7dde943155974"}
Jul 17 11:41:39.642857 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:39.642845    2566 scope.go:117] "RemoveContainer" containerID="fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa"
Jul 17 11:41:40.648730 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:40.648675    2566 generic.go:356] "Generic (PLEG): container finished" podID="97882150-f170-4d9f-95dc-b2c90b79760f" containerID="fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa" exitCode=6
Jul 17 11:41:40.648730 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:40.648724    2566 generic.go:356] "Generic (PLEG): container finished" podID="97882150-f170-4d9f-95dc-b2c90b79760f" containerID="a068c78c4f597e3a79da6c6c0f9899724f910aed55ea3b35a5d6e6766a3165fa" exitCode=6
Jul 17 11:41:40.649217 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:40.648741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz" event={"ID":"97882150-f170-4d9f-95dc-b2c90b79760f","Type":"ContainerDied","Data":"fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa"}
Jul 17 11:41:40.649217 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:40.648835    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz" event={"ID":"97882150-f170-4d9f-95dc-b2c90b79760f","Type":"ContainerDied","Data":"a068c78c4f597e3a79da6c6c0f9899724f910aed55ea3b35a5d6e6766a3165fa"}
Jul 17 11:41:40.649217 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:40.648866    2566 scope.go:117] "RemoveContainer" containerID="fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa"
Jul 17 11:41:40.649217 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:40.649043    2566 scope.go:117] "RemoveContainer" containerID="a068c78c4f597e3a79da6c6c0f9899724f910aed55ea3b35a5d6e6766a3165fa"
Jul 17 11:41:40.649433 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:40.649274    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz_odh-ai-gateway-infra(97882150-f170-4d9f-95dc-b2c90b79760f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz" podUID="97882150-f170-4d9f-95dc-b2c90b79760f"
Jul 17 11:41:40.662078 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:40.662053    2566 scope.go:117] "RemoveContainer" containerID="fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa"
Jul 17 11:41:40.662325 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:40.662303    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa\": container with ID starting with fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa not found: ID does not exist" containerID="fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa"
Jul 17 11:41:40.662406 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:40.662332    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa"} err="failed to get container status \"fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa\": rpc error: code = NotFound desc = could not find container \"fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa\": container with ID starting with fb62c6e7163037d0b816a213ac3489218af20063d1d0197986a1364f3ea8baaa not found: ID does not exist"
Jul 17 11:41:41.655233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:41.655193    2566 scope.go:117] "RemoveContainer" containerID="a068c78c4f597e3a79da6c6c0f9899724f910aed55ea3b35a5d6e6766a3165fa"
Jul 17 11:41:41.655607 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:41.655406    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz_odh-ai-gateway-infra(97882150-f170-4d9f-95dc-b2c90b79760f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz" podUID="97882150-f170-4d9f-95dc-b2c90b79760f"
Jul 17 11:41:47.521416 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.521381    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"]
Jul 17 11:41:47.521802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.521777    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerName="revoke-keys"
Jul 17 11:41:47.521802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.521788    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerName="revoke-keys"
Jul 17 11:41:47.521896 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.521861    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="73672302-ceea-4e40-9bac-ccd8db50d46f" containerName="revoke-keys"
Jul 17 11:41:47.526222 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.526204    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"
Jul 17 11:41:47.533656 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.533127    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"]
Jul 17 11:41:47.687405 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.687372    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb29122b-1ead-42a8-b66d-905e40bc9e7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-vqmxs\" (UID: \"eb29122b-1ead-42a8-b66d-905e40bc9e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"
Jul 17 11:41:47.788647 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.788564    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb29122b-1ead-42a8-b66d-905e40bc9e7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-vqmxs\" (UID: \"eb29122b-1ead-42a8-b66d-905e40bc9e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"
Jul 17 11:41:47.789211 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.789190    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb29122b-1ead-42a8-b66d-905e40bc9e7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-vqmxs\" (UID: \"eb29122b-1ead-42a8-b66d-905e40bc9e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"
Jul 17 11:41:47.837660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:47.837632    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"
Jul 17 11:41:48.177865 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:48.177458    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"]
Jul 17 11:41:48.687643 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:48.687608    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" event={"ID":"eb29122b-1ead-42a8-b66d-905e40bc9e7e","Type":"ContainerStarted","Data":"2c5ce5e860d2839a4c08b1b7b8964aec8329bdc9023218d8fdc158fffe6244d2"}
Jul 17 11:41:48.688087 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:48.687656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" event={"ID":"eb29122b-1ead-42a8-b66d-905e40bc9e7e","Type":"ContainerStarted","Data":"febe2e69784322f69e1c59f58d2c14593cd6cac8e1eeb6b9828d263b2343cede"}
Jul 17 11:41:48.706071 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:48.706023    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" podStartSLOduration=1.70600884 podStartE2EDuration="1.70600884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:41:48.704168191 +0000 UTC m=+2691.561252136" watchObservedRunningTime="2026-07-17 11:41:48.70600884 +0000 UTC m=+2691.563092786"
Jul 17 11:41:49.693682 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:49.693649    2566 generic.go:356] "Generic (PLEG): container finished" podID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerID="2c5ce5e860d2839a4c08b1b7b8964aec8329bdc9023218d8fdc158fffe6244d2" exitCode=6
Jul 17 11:41:49.694172 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:49.693701    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" event={"ID":"eb29122b-1ead-42a8-b66d-905e40bc9e7e","Type":"ContainerDied","Data":"2c5ce5e860d2839a4c08b1b7b8964aec8329bdc9023218d8fdc158fffe6244d2"}
Jul 17 11:41:49.694172 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:49.693927    2566 scope.go:117] "RemoveContainer" containerID="2c5ce5e860d2839a4c08b1b7b8964aec8329bdc9023218d8fdc158fffe6244d2"
Jul 17 11:41:50.700800 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:50.700693    2566 generic.go:356] "Generic (PLEG): container finished" podID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerID="d641ce662f64e12fabf0a0cf4c797ee162c9d0069e1819edeaa4a309cb9223fc" exitCode=6
Jul 17 11:41:50.700800 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:50.700783    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" event={"ID":"eb29122b-1ead-42a8-b66d-905e40bc9e7e","Type":"ContainerDied","Data":"d641ce662f64e12fabf0a0cf4c797ee162c9d0069e1819edeaa4a309cb9223fc"}
Jul 17 11:41:50.701210 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:50.700849    2566 scope.go:117] "RemoveContainer" containerID="2c5ce5e860d2839a4c08b1b7b8964aec8329bdc9023218d8fdc158fffe6244d2"
Jul 17 11:41:50.701210 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:50.701035    2566 scope.go:117] "RemoveContainer" containerID="d641ce662f64e12fabf0a0cf4c797ee162c9d0069e1819edeaa4a309cb9223fc"
Jul 17 11:41:50.701294 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:50.701272    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-vqmxs_odh-ai-gateway-infra(eb29122b-1ead-42a8-b66d-905e40bc9e7e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e"
Jul 17 11:41:51.707148 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:51.707124    2566 scope.go:117] "RemoveContainer" containerID="d641ce662f64e12fabf0a0cf4c797ee162c9d0069e1819edeaa4a309cb9223fc"
Jul 17 11:41:51.707563 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:51.707296    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-vqmxs_odh-ai-gateway-infra(eb29122b-1ead-42a8-b66d-905e40bc9e7e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e"
Jul 17 11:41:52.652269 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:52.652238    2566 scope.go:117] "RemoveContainer" containerID="a068c78c4f597e3a79da6c6c0f9899724f910aed55ea3b35a5d6e6766a3165fa"
Jul 17 11:41:53.720286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:53.718328    2566 generic.go:356] "Generic (PLEG): container finished" podID="97882150-f170-4d9f-95dc-b2c90b79760f" containerID="a9ec392ebc37cac8b0e51c6755370e4b20de9b5def254ed47dd0ab6463fe3579" exitCode=6
Jul 17 11:41:53.720286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:53.718502    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz" event={"ID":"97882150-f170-4d9f-95dc-b2c90b79760f","Type":"ContainerDied","Data":"a9ec392ebc37cac8b0e51c6755370e4b20de9b5def254ed47dd0ab6463fe3579"}
Jul 17 11:41:53.720286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:53.718590    2566 scope.go:117] "RemoveContainer" containerID="a068c78c4f597e3a79da6c6c0f9899724f910aed55ea3b35a5d6e6766a3165fa"
Jul 17 11:41:53.720286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:53.718970    2566 scope.go:117] "RemoveContainer" containerID="a9ec392ebc37cac8b0e51c6755370e4b20de9b5def254ed47dd0ab6463fe3579"
Jul 17 11:41:53.720286 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:41:53.719199    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz_odh-ai-gateway-infra(97882150-f170-4d9f-95dc-b2c90b79760f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz" podUID="97882150-f170-4d9f-95dc-b2c90b79760f"
Jul 17 11:41:54.748445 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:54.748412    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"]
Jul 17 11:41:54.885557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:54.885532    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"
Jul 17 11:41:54.952513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:54.952483    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97882150-f170-4d9f-95dc-b2c90b79760f-maas-api-service-ca\") pod \"97882150-f170-4d9f-95dc-b2c90b79760f\" (UID: \"97882150-f170-4d9f-95dc-b2c90b79760f\") "
Jul 17 11:41:54.952842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:54.952821    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97882150-f170-4d9f-95dc-b2c90b79760f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "97882150-f170-4d9f-95dc-b2c90b79760f" (UID: "97882150-f170-4d9f-95dc-b2c90b79760f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:41:55.054114 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:55.054036    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97882150-f170-4d9f-95dc-b2c90b79760f-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:41:55.731361 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:55.731332    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz" event={"ID":"97882150-f170-4d9f-95dc-b2c90b79760f","Type":"ContainerDied","Data":"112b85a361ce4b7ad79e38fabc0c393c5a8eca0bd0260b32bbd7dde943155974"}
Jul 17 11:41:55.731521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:55.731365    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"
Jul 17 11:41:55.731521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:55.731387    2566 scope.go:117] "RemoveContainer" containerID="a9ec392ebc37cac8b0e51c6755370e4b20de9b5def254ed47dd0ab6463fe3579"
Jul 17 11:41:55.750783 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:55.750758    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"]
Jul 17 11:41:55.755222 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:55.755193    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-pf7jz"]
Jul 17 11:41:57.657494 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:57.657463    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" path="/var/lib/kubelet/pods/97882150-f170-4d9f-95dc-b2c90b79760f/volumes"
Jul 17 11:41:57.847672 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:57.847647    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:41:57.850975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:57.850950    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:41:57.853841 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:57.853818    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:41:57.856781 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:57.856763    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:41:58.763894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.763859    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"]
Jul 17 11:41:58.764389 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.764370    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" containerName="revoke-keys"
Jul 17 11:41:58.764474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.764393    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" containerName="revoke-keys"
Jul 17 11:41:58.764474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.764413    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" containerName="revoke-keys"
Jul 17 11:41:58.764474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.764421    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" containerName="revoke-keys"
Jul 17 11:41:58.764474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.764440    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" containerName="revoke-keys"
Jul 17 11:41:58.764474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.764447    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" containerName="revoke-keys"
Jul 17 11:41:58.764744 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.764554    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" containerName="revoke-keys"
Jul 17 11:41:58.764744 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.764568    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" containerName="revoke-keys"
Jul 17 11:41:58.764744 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.764578    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="97882150-f170-4d9f-95dc-b2c90b79760f" containerName="revoke-keys"
Jul 17 11:41:58.769142 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.769119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"
Jul 17 11:41:58.780505 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.780485    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"]
Jul 17 11:41:58.786392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.786370    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7063226-56a0-4942-b6ee-20334a7ae159-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc\" (UID: \"c7063226-56a0-4942-b6ee-20334a7ae159\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"
Jul 17 11:41:58.887449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.887425    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7063226-56a0-4942-b6ee-20334a7ae159-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc\" (UID: \"c7063226-56a0-4942-b6ee-20334a7ae159\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"
Jul 17 11:41:58.888031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:58.888012    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7063226-56a0-4942-b6ee-20334a7ae159-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc\" (UID: \"c7063226-56a0-4942-b6ee-20334a7ae159\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"
Jul 17 11:41:59.079482 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:59.079455    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"
Jul 17 11:41:59.413870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:59.413840    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"]
Jul 17 11:41:59.415937 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:41:59.415913    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7063226_56a0_4942_b6ee_20334a7ae159.slice/crio-f03cbe1e4c5ad1a0632bf54c68e1590550108fe3307334d37dc730b01b29975b WatchSource:0}: Error finding container f03cbe1e4c5ad1a0632bf54c68e1590550108fe3307334d37dc730b01b29975b: Status 404 returned error can't find the container with id f03cbe1e4c5ad1a0632bf54c68e1590550108fe3307334d37dc730b01b29975b
Jul 17 11:41:59.754542 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:59.754474    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" event={"ID":"c7063226-56a0-4942-b6ee-20334a7ae159","Type":"ContainerStarted","Data":"01d10bf86d795d104dad6604cbf4813223c0d93c8a04a8db5f21fdf731a6db65"}
Jul 17 11:41:59.754542 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:59.754534    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" event={"ID":"c7063226-56a0-4942-b6ee-20334a7ae159","Type":"ContainerStarted","Data":"f03cbe1e4c5ad1a0632bf54c68e1590550108fe3307334d37dc730b01b29975b"}
Jul 17 11:41:59.774889 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:41:59.774839    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" podStartSLOduration=1.7748227619999999 podStartE2EDuration="1.774822762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:41:59.773655825 +0000 UTC m=+2702.630739769" watchObservedRunningTime="2026-07-17 11:41:59.774822762 +0000 UTC m=+2702.631906707"
Jul 17 11:42:00.760924 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:00.760889    2566 generic.go:356] "Generic (PLEG): container finished" podID="c7063226-56a0-4942-b6ee-20334a7ae159" containerID="01d10bf86d795d104dad6604cbf4813223c0d93c8a04a8db5f21fdf731a6db65" exitCode=6
Jul 17 11:42:00.761122 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:00.760934    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" event={"ID":"c7063226-56a0-4942-b6ee-20334a7ae159","Type":"ContainerDied","Data":"01d10bf86d795d104dad6604cbf4813223c0d93c8a04a8db5f21fdf731a6db65"}
Jul 17 11:42:00.761196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:00.761183    2566 scope.go:117] "RemoveContainer" containerID="01d10bf86d795d104dad6604cbf4813223c0d93c8a04a8db5f21fdf731a6db65"
Jul 17 11:42:01.769552 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:01.769471    2566 generic.go:356] "Generic (PLEG): container finished" podID="c7063226-56a0-4942-b6ee-20334a7ae159" containerID="abee5059f42a37550a48847bba3172ae4f194640c729532f783cff7871e0872e" exitCode=6
Jul 17 11:42:01.769957 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:01.769554    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" event={"ID":"c7063226-56a0-4942-b6ee-20334a7ae159","Type":"ContainerDied","Data":"abee5059f42a37550a48847bba3172ae4f194640c729532f783cff7871e0872e"}
Jul 17 11:42:01.769957 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:01.769614    2566 scope.go:117] "RemoveContainer" containerID="01d10bf86d795d104dad6604cbf4813223c0d93c8a04a8db5f21fdf731a6db65"
Jul 17 11:42:01.769957 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:01.769698    2566 scope.go:117] "RemoveContainer" containerID="abee5059f42a37550a48847bba3172ae4f194640c729532f783cff7871e0872e"
Jul 17 11:42:01.769957 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:01.769911    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc_odh-ai-gateway-infra(c7063226-56a0-4942-b6ee-20334a7ae159)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" podUID="c7063226-56a0-4942-b6ee-20334a7ae159"
Jul 17 11:42:02.651858 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:02.651827    2566 scope.go:117] "RemoveContainer" containerID="d641ce662f64e12fabf0a0cf4c797ee162c9d0069e1819edeaa4a309cb9223fc"
Jul 17 11:42:02.777322 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:02.777293    2566 scope.go:117] "RemoveContainer" containerID="abee5059f42a37550a48847bba3172ae4f194640c729532f783cff7871e0872e"
Jul 17 11:42:02.777673 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:02.777492    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc_odh-ai-gateway-infra(c7063226-56a0-4942-b6ee-20334a7ae159)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" podUID="c7063226-56a0-4942-b6ee-20334a7ae159"
Jul 17 11:42:03.784219 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:03.784190    2566 generic.go:356] "Generic (PLEG): container finished" podID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerID="9e47602f7a5b3f26e6f31c24e3a949afd8483618adb6cc168a6ca885a4f06d80" exitCode=6
Jul 17 11:42:03.784653 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:03.784241    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" event={"ID":"eb29122b-1ead-42a8-b66d-905e40bc9e7e","Type":"ContainerDied","Data":"9e47602f7a5b3f26e6f31c24e3a949afd8483618adb6cc168a6ca885a4f06d80"}
Jul 17 11:42:03.784653 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:03.784291    2566 scope.go:117] "RemoveContainer" containerID="d641ce662f64e12fabf0a0cf4c797ee162c9d0069e1819edeaa4a309cb9223fc"
Jul 17 11:42:03.784653 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:03.784599    2566 scope.go:117] "RemoveContainer" containerID="9e47602f7a5b3f26e6f31c24e3a949afd8483618adb6cc168a6ca885a4f06d80"
Jul 17 11:42:03.784815 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:03.784794    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-vqmxs_odh-ai-gateway-infra(eb29122b-1ead-42a8-b66d-905e40bc9e7e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e"
Jul 17 11:42:04.814588 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:04.814559    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"]
Jul 17 11:42:04.944357 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:04.944335    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"
Jul 17 11:42:05.045992 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:05.045953    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb29122b-1ead-42a8-b66d-905e40bc9e7e-maas-api-service-ca\") pod \"eb29122b-1ead-42a8-b66d-905e40bc9e7e\" (UID: \"eb29122b-1ead-42a8-b66d-905e40bc9e7e\") "
Jul 17 11:42:05.046337 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:05.046313    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb29122b-1ead-42a8-b66d-905e40bc9e7e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb29122b-1ead-42a8-b66d-905e40bc9e7e" (UID: "eb29122b-1ead-42a8-b66d-905e40bc9e7e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:42:05.147104 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:05.147077    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb29122b-1ead-42a8-b66d-905e40bc9e7e-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:05.797675 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:05.797638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs" event={"ID":"eb29122b-1ead-42a8-b66d-905e40bc9e7e","Type":"ContainerDied","Data":"febe2e69784322f69e1c59f58d2c14593cd6cac8e1eeb6b9828d263b2343cede"}
Jul 17 11:42:05.797870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:05.797652    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"
Jul 17 11:42:05.797870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:05.797733    2566 scope.go:117] "RemoveContainer" containerID="9e47602f7a5b3f26e6f31c24e3a949afd8483618adb6cc168a6ca885a4f06d80"
Jul 17 11:42:05.817246 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:05.817213    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"]
Jul 17 11:42:05.821762 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:05.821740    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-vqmxs"]
Jul 17 11:42:07.657855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:07.657822    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" path="/var/lib/kubelet/pods/eb29122b-1ead-42a8-b66d-905e40bc9e7e/volumes"
Jul 17 11:42:14.652466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:14.652435    2566 scope.go:117] "RemoveContainer" containerID="abee5059f42a37550a48847bba3172ae4f194640c729532f783cff7871e0872e"
Jul 17 11:42:15.848350 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:15.848322    2566 generic.go:356] "Generic (PLEG): container finished" podID="c7063226-56a0-4942-b6ee-20334a7ae159" containerID="cd5dd4be3d10c3a032cd1865747c637019ec917356a3aaadf6e5687c3e43d982" exitCode=6
Jul 17 11:42:15.848745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:15.848394    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" event={"ID":"c7063226-56a0-4942-b6ee-20334a7ae159","Type":"ContainerDied","Data":"cd5dd4be3d10c3a032cd1865747c637019ec917356a3aaadf6e5687c3e43d982"}
Jul 17 11:42:15.848745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:15.848456    2566 scope.go:117] "RemoveContainer" containerID="abee5059f42a37550a48847bba3172ae4f194640c729532f783cff7871e0872e"
Jul 17 11:42:15.848745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:15.848741    2566 scope.go:117] "RemoveContainer" containerID="cd5dd4be3d10c3a032cd1865747c637019ec917356a3aaadf6e5687c3e43d982"
Jul 17 11:42:15.848964 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:15.848945    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc_odh-ai-gateway-infra(c7063226-56a0-4942-b6ee-20334a7ae159)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" podUID="c7063226-56a0-4942-b6ee-20334a7ae159"
Jul 17 11:42:16.878233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:16.878202    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"]
Jul 17 11:42:17.011113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:17.011085    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"
Jul 17 11:42:17.155458 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:17.155389    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7063226-56a0-4942-b6ee-20334a7ae159-maas-api-service-ca\") pod \"c7063226-56a0-4942-b6ee-20334a7ae159\" (UID: \"c7063226-56a0-4942-b6ee-20334a7ae159\") "
Jul 17 11:42:17.155696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:17.155678    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7063226-56a0-4942-b6ee-20334a7ae159-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c7063226-56a0-4942-b6ee-20334a7ae159" (UID: "c7063226-56a0-4942-b6ee-20334a7ae159"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:42:17.256588 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:17.256542    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7063226-56a0-4942-b6ee-20334a7ae159-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:17.861553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:17.861527    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"
Jul 17 11:42:17.861763 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:17.861566    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc" event={"ID":"c7063226-56a0-4942-b6ee-20334a7ae159","Type":"ContainerDied","Data":"f03cbe1e4c5ad1a0632bf54c68e1590550108fe3307334d37dc730b01b29975b"}
Jul 17 11:42:17.861763 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:17.861624    2566 scope.go:117] "RemoveContainer" containerID="cd5dd4be3d10c3a032cd1865747c637019ec917356a3aaadf6e5687c3e43d982"
Jul 17 11:42:17.882291 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:17.882268    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"]
Jul 17 11:42:17.885218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:17.885196    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cpzvc"]
Jul 17 11:42:19.657827 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:19.657792    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" path="/var/lib/kubelet/pods/c7063226-56a0-4942-b6ee-20334a7ae159/volumes"
Jul 17 11:42:28.700898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.700863    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"]
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701238    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701252    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701269    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701276    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701284    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701289    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701301    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701307    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701315    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701320    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701328    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerName="revoke-keys"
Jul 17 11:42:28.701392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701333    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerName="revoke-keys"
Jul 17 11:42:28.701852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701401    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerName="revoke-keys"
Jul 17 11:42:28.701852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701408    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" containerName="revoke-keys"
Jul 17 11:42:28.701852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701415    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" containerName="revoke-keys"
Jul 17 11:42:28.701852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.701425    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerName="revoke-keys"
Jul 17 11:42:28.705595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.705576    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"
Jul 17 11:42:28.708427 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.708403    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:42:28.708552 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.708405    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:42:28.713539 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.713514    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"]
Jul 17 11:42:28.859397 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.859363    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0f2abd-6201-443a-a71c-68457975aca7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g\" (UID: \"3a0f2abd-6201-443a-a71c-68457975aca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"
Jul 17 11:42:28.960822 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.960746    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0f2abd-6201-443a-a71c-68457975aca7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g\" (UID: \"3a0f2abd-6201-443a-a71c-68457975aca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"
Jul 17 11:42:28.961340 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:28.961318    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0f2abd-6201-443a-a71c-68457975aca7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g\" (UID: \"3a0f2abd-6201-443a-a71c-68457975aca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"
Jul 17 11:42:29.016946 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:29.016915    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"
Jul 17 11:42:29.146385 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:29.146361    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"]
Jul 17 11:42:29.148689 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:42:29.148660    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a0f2abd_6201_443a_a71c_68457975aca7.slice/crio-571b142179c772a479cdd9014621a9e57923ab42920e910791872c49f97f84a2 WatchSource:0}: Error finding container 571b142179c772a479cdd9014621a9e57923ab42920e910791872c49f97f84a2: Status 404 returned error can't find the container with id 571b142179c772a479cdd9014621a9e57923ab42920e910791872c49f97f84a2
Jul 17 11:42:29.921779 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:29.921749    2566 generic.go:356] "Generic (PLEG): container finished" podID="3a0f2abd-6201-443a-a71c-68457975aca7" containerID="09c449e5df9eab709cb163241545f1d7c26c956cdf9203420a062df965625c83" exitCode=6
Jul 17 11:42:29.921779 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:29.921784    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" event={"ID":"3a0f2abd-6201-443a-a71c-68457975aca7","Type":"ContainerDied","Data":"09c449e5df9eab709cb163241545f1d7c26c956cdf9203420a062df965625c83"}
Jul 17 11:42:29.922200 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:29.921820    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" event={"ID":"3a0f2abd-6201-443a-a71c-68457975aca7","Type":"ContainerStarted","Data":"571b142179c772a479cdd9014621a9e57923ab42920e910791872c49f97f84a2"}
Jul 17 11:42:29.922200 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:29.922034    2566 scope.go:117] "RemoveContainer" containerID="09c449e5df9eab709cb163241545f1d7c26c956cdf9203420a062df965625c83"
Jul 17 11:42:30.929275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:30.929182    2566 generic.go:356] "Generic (PLEG): container finished" podID="3a0f2abd-6201-443a-a71c-68457975aca7" containerID="25f3806fe403eb9e65e65fc01d830aced5713348bda0893bc807f08ce2a30674" exitCode=6
Jul 17 11:42:30.929275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:30.929225    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" event={"ID":"3a0f2abd-6201-443a-a71c-68457975aca7","Type":"ContainerDied","Data":"25f3806fe403eb9e65e65fc01d830aced5713348bda0893bc807f08ce2a30674"}
Jul 17 11:42:30.929777 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:30.929288    2566 scope.go:117] "RemoveContainer" containerID="09c449e5df9eab709cb163241545f1d7c26c956cdf9203420a062df965625c83"
Jul 17 11:42:30.929777 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:30.929432    2566 scope.go:117] "RemoveContainer" containerID="25f3806fe403eb9e65e65fc01d830aced5713348bda0893bc807f08ce2a30674"
Jul 17 11:42:30.929777 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:30.929685    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g_odh-ai-gateway-infra(3a0f2abd-6201-443a-a71c-68457975aca7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" podUID="3a0f2abd-6201-443a-a71c-68457975aca7"
Jul 17 11:42:31.935670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:31.935639    2566 scope.go:117] "RemoveContainer" containerID="25f3806fe403eb9e65e65fc01d830aced5713348bda0893bc807f08ce2a30674"
Jul 17 11:42:31.936164 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:31.935911    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g_odh-ai-gateway-infra(3a0f2abd-6201-443a-a71c-68457975aca7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" podUID="3a0f2abd-6201-443a-a71c-68457975aca7"
Jul 17 11:42:35.468365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.468331    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"]
Jul 17 11:42:35.468828 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.468812    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb29122b-1ead-42a8-b66d-905e40bc9e7e" containerName="revoke-keys"
Jul 17 11:42:35.468886 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.468832    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7063226-56a0-4942-b6ee-20334a7ae159" containerName="revoke-keys"
Jul 17 11:42:35.473058 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.473043    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"
Jul 17 11:42:35.481337 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.481313    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"]
Jul 17 11:42:35.622965 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.622935    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23f6ab4-8f76-44c1-9382-d677e49e6e5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-7989h\" (UID: \"d23f6ab4-8f76-44c1-9382-d677e49e6e5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"
Jul 17 11:42:35.724107 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.724026    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23f6ab4-8f76-44c1-9382-d677e49e6e5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-7989h\" (UID: \"d23f6ab4-8f76-44c1-9382-d677e49e6e5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"
Jul 17 11:42:35.724629 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.724604    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23f6ab4-8f76-44c1-9382-d677e49e6e5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-7989h\" (UID: \"d23f6ab4-8f76-44c1-9382-d677e49e6e5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"
Jul 17 11:42:35.784495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.784467    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"
Jul 17 11:42:35.918141 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.918118    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"]
Jul 17 11:42:35.919846 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:42:35.919815    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23f6ab4_8f76_44c1_9382_d677e49e6e5e.slice/crio-5ad0e41006ccf92e7da6cc181526efbdd22db90a05709b761655f062391264ba WatchSource:0}: Error finding container 5ad0e41006ccf92e7da6cc181526efbdd22db90a05709b761655f062391264ba: Status 404 returned error can't find the container with id 5ad0e41006ccf92e7da6cc181526efbdd22db90a05709b761655f062391264ba
Jul 17 11:42:35.955030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:35.955005    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" event={"ID":"d23f6ab4-8f76-44c1-9382-d677e49e6e5e","Type":"ContainerStarted","Data":"5ad0e41006ccf92e7da6cc181526efbdd22db90a05709b761655f062391264ba"}
Jul 17 11:42:36.787819 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:36.787740    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"]
Jul 17 11:42:36.788218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:36.787945    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy" containerID="cri-o://3cfbe0824af2c7042ee8bf97067b6168f2324f0096bc2d75da8627e659a3584c" gracePeriod=30
Jul 17 11:42:36.961926 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:36.961885    2566 generic.go:356] "Generic (PLEG): container finished" podID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerID="7493cad1cf0fd9074ca7f138df6cac0adb916229e9908b03ae56a5a0c9c12f4a" exitCode=6
Jul 17 11:42:36.962111 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:36.961933    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" event={"ID":"d23f6ab4-8f76-44c1-9382-d677e49e6e5e","Type":"ContainerDied","Data":"7493cad1cf0fd9074ca7f138df6cac0adb916229e9908b03ae56a5a0c9c12f4a"}
Jul 17 11:42:36.962171 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:36.962165    2566 scope.go:117] "RemoveContainer" containerID="7493cad1cf0fd9074ca7f138df6cac0adb916229e9908b03ae56a5a0c9c12f4a"
Jul 17 11:42:37.969347 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:37.969312    2566 generic.go:356] "Generic (PLEG): container finished" podID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerID="a508efc51a3f4f0e63096c223fbfe396f678b5c7be958a4a273e212dd5b6b5a1" exitCode=6
Jul 17 11:42:37.969800 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:37.969384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" event={"ID":"d23f6ab4-8f76-44c1-9382-d677e49e6e5e","Type":"ContainerDied","Data":"a508efc51a3f4f0e63096c223fbfe396f678b5c7be958a4a273e212dd5b6b5a1"}
Jul 17 11:42:37.969800 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:37.969426    2566 scope.go:117] "RemoveContainer" containerID="7493cad1cf0fd9074ca7f138df6cac0adb916229e9908b03ae56a5a0c9c12f4a"
Jul 17 11:42:37.969800 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:37.969603    2566 scope.go:117] "RemoveContainer" containerID="a508efc51a3f4f0e63096c223fbfe396f678b5c7be958a4a273e212dd5b6b5a1"
Jul 17 11:42:37.969935 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:37.969859    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-7989h_odh-ai-gateway-infra(d23f6ab4-8f76-44c1-9382-d677e49e6e5e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e"
Jul 17 11:42:38.975666 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:38.975640    2566 scope.go:117] "RemoveContainer" containerID="a508efc51a3f4f0e63096c223fbfe396f678b5c7be958a4a273e212dd5b6b5a1"
Jul 17 11:42:38.976034 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:38.975844    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-7989h_odh-ai-gateway-infra(d23f6ab4-8f76-44c1-9382-d677e49e6e5e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e"
Jul 17 11:42:39.720696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:39.720661    2566 patch_prober.go:28] interesting pod/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 11:42:39.720862 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:39.720739    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 11:42:41.369247 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:41.369212    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:42:41.992043 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:41.992003    2566 generic.go:356] "Generic (PLEG): container finished" podID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerID="3cfbe0824af2c7042ee8bf97067b6168f2324f0096bc2d75da8627e659a3584c" exitCode=0
Jul 17 11:42:41.992218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:41.992069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" event={"ID":"c74c80eb-4ad6-431c-b227-2669d494ec2b","Type":"ContainerDied","Data":"3cfbe0824af2c7042ee8bf97067b6168f2324f0096bc2d75da8627e659a3584c"}
Jul 17 11:42:41.992218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:41.992150    2566 scope.go:117] "RemoveContainer" containerID="736a7f14639bfe4a53e8cb2f21a4206397115a5d98f5c34c0d2705f6062f7c72"
Jul 17 11:42:42.044393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.044373    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:42:42.073333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073307    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c74c80eb-4ad6-431c-b227-2669d494ec2b-istiod-ca-cert\") pod \"c74c80eb-4ad6-431c-b227-2669d494ec2b\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") "
Jul 17 11:42:42.073508 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073348    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-certs\") pod \"c74c80eb-4ad6-431c-b227-2669d494ec2b\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") "
Jul 17 11:42:42.073508 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073369    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-credential-socket\") pod \"c74c80eb-4ad6-431c-b227-2669d494ec2b\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") "
Jul 17 11:42:42.073508 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073397    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-data\") pod \"c74c80eb-4ad6-431c-b227-2669d494ec2b\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") "
Jul 17 11:42:42.073689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073664    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c74c80eb-4ad6-431c-b227-2669d494ec2b" (UID: "c74c80eb-4ad6-431c-b227-2669d494ec2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:42:42.073761 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073733    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-envoy\") pod \"c74c80eb-4ad6-431c-b227-2669d494ec2b\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") "
Jul 17 11:42:42.073799 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073776    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-token\") pod \"c74c80eb-4ad6-431c-b227-2669d494ec2b\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") "
Jul 17 11:42:42.073842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073796    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c74c80eb-4ad6-431c-b227-2669d494ec2b" (UID: "c74c80eb-4ad6-431c-b227-2669d494ec2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:42:42.073842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073808    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-socket\") pod \"c74c80eb-4ad6-431c-b227-2669d494ec2b\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") "
Jul 17 11:42:42.073925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073801    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c74c80eb-4ad6-431c-b227-2669d494ec2b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c74c80eb-4ad6-431c-b227-2669d494ec2b" (UID: "c74c80eb-4ad6-431c-b227-2669d494ec2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:42:42.073925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073873    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql5wv\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-kube-api-access-ql5wv\") pod \"c74c80eb-4ad6-431c-b227-2669d494ec2b\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") "
Jul 17 11:42:42.073925 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073911    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-podinfo\") pod \"c74c80eb-4ad6-431c-b227-2669d494ec2b\" (UID: \"c74c80eb-4ad6-431c-b227-2669d494ec2b\") "
Jul 17 11:42:42.074086 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.073994    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c74c80eb-4ad6-431c-b227-2669d494ec2b" (UID: "c74c80eb-4ad6-431c-b227-2669d494ec2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:42:42.074239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.074212    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-data" (OuterVolumeSpecName: "istio-data") pod "c74c80eb-4ad6-431c-b227-2669d494ec2b" (UID: "c74c80eb-4ad6-431c-b227-2669d494ec2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:42:42.074331 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.074253    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:42.074331 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.074272    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c74c80eb-4ad6-431c-b227-2669d494ec2b-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:42.074331 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.074286    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:42.074331 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.074300    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:42.076046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.076020    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c74c80eb-4ad6-431c-b227-2669d494ec2b" (UID: "c74c80eb-4ad6-431c-b227-2669d494ec2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:42:42.076046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.076039    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c74c80eb-4ad6-431c-b227-2669d494ec2b" (UID: "c74c80eb-4ad6-431c-b227-2669d494ec2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:42:42.076234 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.076039    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-kube-api-access-ql5wv" (OuterVolumeSpecName: "kube-api-access-ql5wv") pod "c74c80eb-4ad6-431c-b227-2669d494ec2b" (UID: "c74c80eb-4ad6-431c-b227-2669d494ec2b"). InnerVolumeSpecName "kube-api-access-ql5wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:42:42.076234 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.076218    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-token" (OuterVolumeSpecName: "istio-token") pod "c74c80eb-4ad6-431c-b227-2669d494ec2b" (UID: "c74c80eb-4ad6-431c-b227-2669d494ec2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:42:42.175366 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.175333    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:42.175366 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.175361    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:42.175366 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.175370    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:42.175599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.175380    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql5wv\" (UniqueName: \"kubernetes.io/projected/c74c80eb-4ad6-431c-b227-2669d494ec2b-kube-api-access-ql5wv\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:42.175599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.175389    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c74c80eb-4ad6-431c-b227-2669d494ec2b-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:42.999077 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.999046    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p" event={"ID":"c74c80eb-4ad6-431c-b227-2669d494ec2b","Type":"ContainerDied","Data":"78d343a93ff1d12a1a41f47eef60aaa4971f8062069566fd87f4bde492f10498"}
Jul 17 11:42:42.999077 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.999077    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"
Jul 17 11:42:42.999612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:42.999111    2566 scope.go:117] "RemoveContainer" containerID="3cfbe0824af2c7042ee8bf97067b6168f2324f0096bc2d75da8627e659a3584c"
Jul 17 11:42:43.028866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:43.028836    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"]
Jul 17 11:42:43.033516 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:43.033493    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-63359af2-openshift-default-d6cbd5bbc-rhw2p"]
Jul 17 11:42:43.664393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:43.664354    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" path="/var/lib/kubelet/pods/c74c80eb-4ad6-431c-b227-2669d494ec2b/volumes"
Jul 17 11:42:43.665273 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:43.665248    2566 scope.go:117] "RemoveContainer" containerID="25f3806fe403eb9e65e65fc01d830aced5713348bda0893bc807f08ce2a30674"
Jul 17 11:42:44.007748 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:44.007685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" event={"ID":"3a0f2abd-6201-443a-a71c-68457975aca7","Type":"ContainerStarted","Data":"4417d35662837f6c060df8edb269b99cce9e0e834366f7fd1c585f0bd63576e5"}
Jul 17 11:42:44.027016 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:44.026969    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" podStartSLOduration=16.026955821 podStartE2EDuration="16.026955821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:42:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:42:44.026196657 +0000 UTC m=+2746.883280602" watchObservedRunningTime="2026-07-17 11:42:44.026955821 +0000 UTC m=+2746.884039766"
Jul 17 11:42:45.015793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:45.015761    2566 generic.go:356] "Generic (PLEG): container finished" podID="3a0f2abd-6201-443a-a71c-68457975aca7" containerID="4417d35662837f6c060df8edb269b99cce9e0e834366f7fd1c585f0bd63576e5" exitCode=6
Jul 17 11:42:45.016265 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:45.015833    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" event={"ID":"3a0f2abd-6201-443a-a71c-68457975aca7","Type":"ContainerDied","Data":"4417d35662837f6c060df8edb269b99cce9e0e834366f7fd1c585f0bd63576e5"}
Jul 17 11:42:45.016265 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:45.015893    2566 scope.go:117] "RemoveContainer" containerID="25f3806fe403eb9e65e65fc01d830aced5713348bda0893bc807f08ce2a30674"
Jul 17 11:42:45.016265 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:45.016145    2566 scope.go:117] "RemoveContainer" containerID="4417d35662837f6c060df8edb269b99cce9e0e834366f7fd1c585f0bd63576e5"
Jul 17 11:42:45.016385 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:45.016359    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g_odh-ai-gateway-infra(3a0f2abd-6201-443a-a71c-68457975aca7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" podUID="3a0f2abd-6201-443a-a71c-68457975aca7"
Jul 17 11:42:45.043948 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:45.043926    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"]
Jul 17 11:42:46.155700 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:46.155676    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"
Jul 17 11:42:46.211984 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:46.211958    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0f2abd-6201-443a-a71c-68457975aca7-maas-api-service-ca\") pod \"3a0f2abd-6201-443a-a71c-68457975aca7\" (UID: \"3a0f2abd-6201-443a-a71c-68457975aca7\") "
Jul 17 11:42:46.212281 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:46.212261    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a0f2abd-6201-443a-a71c-68457975aca7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a0f2abd-6201-443a-a71c-68457975aca7" (UID: "3a0f2abd-6201-443a-a71c-68457975aca7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:42:46.312910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:46.312838    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0f2abd-6201-443a-a71c-68457975aca7-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:47.027673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:47.027639    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"
Jul 17 11:42:47.027867 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:47.027677    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g" event={"ID":"3a0f2abd-6201-443a-a71c-68457975aca7","Type":"ContainerDied","Data":"571b142179c772a479cdd9014621a9e57923ab42920e910791872c49f97f84a2"}
Jul 17 11:42:47.027867 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:47.027762    2566 scope.go:117] "RemoveContainer" containerID="4417d35662837f6c060df8edb269b99cce9e0e834366f7fd1c585f0bd63576e5"
Jul 17 11:42:47.062160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:47.062132    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"]
Jul 17 11:42:47.077048 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:47.077022    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-tpk7g"]
Jul 17 11:42:47.657014 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:47.656982    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" path="/var/lib/kubelet/pods/3a0f2abd-6201-443a-a71c-68457975aca7/volumes"
Jul 17 11:42:48.798027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.797992    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"]
Jul 17 11:42:48.798417 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798369    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy"
Jul 17 11:42:48.798417 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798380    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy"
Jul 17 11:42:48.798417 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798390    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" containerName="revoke-keys"
Jul 17 11:42:48.798417 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798396    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" containerName="revoke-keys"
Jul 17 11:42:48.798417 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798407    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" containerName="revoke-keys"
Jul 17 11:42:48.798417 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798413    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" containerName="revoke-keys"
Jul 17 11:42:48.798599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798426    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" containerName="revoke-keys"
Jul 17 11:42:48.798599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798432    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" containerName="revoke-keys"
Jul 17 11:42:48.798599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798481    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy"
Jul 17 11:42:48.798599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798490    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" containerName="revoke-keys"
Jul 17 11:42:48.798599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.798496    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" containerName="revoke-keys"
Jul 17 11:42:48.802912 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.802894    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"
Jul 17 11:42:48.813210 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.813177    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"]
Jul 17 11:42:48.833334 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.833306    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab12ff7-aab0-467d-ab00-68156601edf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb\" (UID: \"5ab12ff7-aab0-467d-ab00-68156601edf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"
Jul 17 11:42:48.934142 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.934104    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab12ff7-aab0-467d-ab00-68156601edf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb\" (UID: \"5ab12ff7-aab0-467d-ab00-68156601edf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"
Jul 17 11:42:48.934727 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:48.934687    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab12ff7-aab0-467d-ab00-68156601edf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb\" (UID: \"5ab12ff7-aab0-467d-ab00-68156601edf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"
Jul 17 11:42:49.114164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:49.114132    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"
Jul 17 11:42:49.246376 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:49.246354    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"]
Jul 17 11:42:50.044574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:50.044539    2566 generic.go:356] "Generic (PLEG): container finished" podID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerID="17cfec7ceb52d116bca19bf8bdf2e6ed58699481c0de92884a150c65f7abf4ff" exitCode=6
Jul 17 11:42:50.045005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:50.044623    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" event={"ID":"5ab12ff7-aab0-467d-ab00-68156601edf7","Type":"ContainerDied","Data":"17cfec7ceb52d116bca19bf8bdf2e6ed58699481c0de92884a150c65f7abf4ff"}
Jul 17 11:42:50.045005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:50.044679    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" event={"ID":"5ab12ff7-aab0-467d-ab00-68156601edf7","Type":"ContainerStarted","Data":"bc5b5c00756739b5caf3811ce0a11f6d8647cd6dcb4335b687208c9ea68a45e0"}
Jul 17 11:42:50.045005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:50.044823    2566 scope.go:117] "RemoveContainer" containerID="17cfec7ceb52d116bca19bf8bdf2e6ed58699481c0de92884a150c65f7abf4ff"
Jul 17 11:42:50.651647 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:50.651621    2566 scope.go:117] "RemoveContainer" containerID="a508efc51a3f4f0e63096c223fbfe396f678b5c7be958a4a273e212dd5b6b5a1"
Jul 17 11:42:51.051575 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:51.051486    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" event={"ID":"d23f6ab4-8f76-44c1-9382-d677e49e6e5e","Type":"ContainerStarted","Data":"8a53634bccd12e79cb619a034327f3a211d26c89e94096a5f71993b332f5aeee"}
Jul 17 11:42:51.053240 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:51.053214    2566 generic.go:356] "Generic (PLEG): container finished" podID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerID="89391dbf709ece4d39deb0d31a4b376e6ab347ccec0d3946f9cce9f09aac8cae" exitCode=6
Jul 17 11:42:51.053350 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:51.053255    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" event={"ID":"5ab12ff7-aab0-467d-ab00-68156601edf7","Type":"ContainerDied","Data":"89391dbf709ece4d39deb0d31a4b376e6ab347ccec0d3946f9cce9f09aac8cae"}
Jul 17 11:42:51.053350 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:51.053304    2566 scope.go:117] "RemoveContainer" containerID="17cfec7ceb52d116bca19bf8bdf2e6ed58699481c0de92884a150c65f7abf4ff"
Jul 17 11:42:51.053439 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:51.053414    2566 scope.go:117] "RemoveContainer" containerID="89391dbf709ece4d39deb0d31a4b376e6ab347ccec0d3946f9cce9f09aac8cae"
Jul 17 11:42:51.053621 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:51.053602    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb_odh-ai-gateway-infra(5ab12ff7-aab0-467d-ab00-68156601edf7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7"
Jul 17 11:42:51.075528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:51.075480    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" podStartSLOduration=16.075467709 podStartE2EDuration="16.075467709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:42:51.075142998 +0000 UTC m=+2753.932226942" watchObservedRunningTime="2026-07-17 11:42:51.075467709 +0000 UTC m=+2753.932551635"
Jul 17 11:42:52.060167 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:52.060135    2566 generic.go:356] "Generic (PLEG): container finished" podID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerID="8a53634bccd12e79cb619a034327f3a211d26c89e94096a5f71993b332f5aeee" exitCode=6
Jul 17 11:42:52.060645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:52.060212    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" event={"ID":"d23f6ab4-8f76-44c1-9382-d677e49e6e5e","Type":"ContainerDied","Data":"8a53634bccd12e79cb619a034327f3a211d26c89e94096a5f71993b332f5aeee"}
Jul 17 11:42:52.060645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:52.060272    2566 scope.go:117] "RemoveContainer" containerID="a508efc51a3f4f0e63096c223fbfe396f678b5c7be958a4a273e212dd5b6b5a1"
Jul 17 11:42:52.060645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:52.060564    2566 scope.go:117] "RemoveContainer" containerID="8a53634bccd12e79cb619a034327f3a211d26c89e94096a5f71993b332f5aeee"
Jul 17 11:42:52.060834 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:52.060809    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-7989h_odh-ai-gateway-infra(d23f6ab4-8f76-44c1-9382-d677e49e6e5e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e"
Jul 17 11:42:52.062038 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:52.062021    2566 scope.go:117] "RemoveContainer" containerID="89391dbf709ece4d39deb0d31a4b376e6ab347ccec0d3946f9cce9f09aac8cae"
Jul 17 11:42:52.062180 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:42:52.062162    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb_odh-ai-gateway-infra(5ab12ff7-aab0-467d-ab00-68156601edf7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7"
Jul 17 11:42:52.086606 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:52.086585    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"]
Jul 17 11:42:53.207224 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:53.207196    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"
Jul 17 11:42:53.269250 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:53.269223    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23f6ab4-8f76-44c1-9382-d677e49e6e5e-maas-api-service-ca\") pod \"d23f6ab4-8f76-44c1-9382-d677e49e6e5e\" (UID: \"d23f6ab4-8f76-44c1-9382-d677e49e6e5e\") "
Jul 17 11:42:53.269596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:53.269574    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d23f6ab4-8f76-44c1-9382-d677e49e6e5e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d23f6ab4-8f76-44c1-9382-d677e49e6e5e" (UID: "d23f6ab4-8f76-44c1-9382-d677e49e6e5e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:42:53.370536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:53.370495    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23f6ab4-8f76-44c1-9382-d677e49e6e5e-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:42:54.074714 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:54.074687    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"
Jul 17 11:42:54.074866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:54.074687    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h" event={"ID":"d23f6ab4-8f76-44c1-9382-d677e49e6e5e","Type":"ContainerDied","Data":"5ad0e41006ccf92e7da6cc181526efbdd22db90a05709b761655f062391264ba"}
Jul 17 11:42:54.074866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:54.074839    2566 scope.go:117] "RemoveContainer" containerID="8a53634bccd12e79cb619a034327f3a211d26c89e94096a5f71993b332f5aeee"
Jul 17 11:42:54.095011 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:54.094986    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"]
Jul 17 11:42:54.102900 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:54.102873    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-7989h"]
Jul 17 11:42:55.657701 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:55.657663    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" path="/var/lib/kubelet/pods/d23f6ab4-8f76-44c1-9382-d677e49e6e5e/volumes"
Jul 17 11:42:56.657966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:42:56.657936    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:43:03.652302 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:03.652273    2566 scope.go:117] "RemoveContainer" containerID="89391dbf709ece4d39deb0d31a4b376e6ab347ccec0d3946f9cce9f09aac8cae"
Jul 17 11:43:04.123714 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:04.123680    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" event={"ID":"5ab12ff7-aab0-467d-ab00-68156601edf7","Type":"ContainerStarted","Data":"228b49c1e5f1400c7854c14e5bd005812d3ec147567ec03f47ad093713ce88bf"}
Jul 17 11:43:04.146780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:04.146731    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" podStartSLOduration=16.146700338 podStartE2EDuration="16.146700338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:43:04.144651844 +0000 UTC m=+2767.001735805" watchObservedRunningTime="2026-07-17 11:43:04.146700338 +0000 UTC m=+2767.003784283"
Jul 17 11:43:05.130348 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:05.130313    2566 generic.go:356] "Generic (PLEG): container finished" podID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerID="228b49c1e5f1400c7854c14e5bd005812d3ec147567ec03f47ad093713ce88bf" exitCode=6
Jul 17 11:43:05.130785 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:05.130356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" event={"ID":"5ab12ff7-aab0-467d-ab00-68156601edf7","Type":"ContainerDied","Data":"228b49c1e5f1400c7854c14e5bd005812d3ec147567ec03f47ad093713ce88bf"}
Jul 17 11:43:05.130785 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:05.130427    2566 scope.go:117] "RemoveContainer" containerID="89391dbf709ece4d39deb0d31a4b376e6ab347ccec0d3946f9cce9f09aac8cae"
Jul 17 11:43:05.130889 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:05.130783    2566 scope.go:117] "RemoveContainer" containerID="228b49c1e5f1400c7854c14e5bd005812d3ec147567ec03f47ad093713ce88bf"
Jul 17 11:43:05.131010 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:05.130982    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb_odh-ai-gateway-infra(5ab12ff7-aab0-467d-ab00-68156601edf7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7"
Jul 17 11:43:05.161499 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:05.161475    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"]
Jul 17 11:43:06.271930 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:06.271907    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"
Jul 17 11:43:06.378859 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:06.378834    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab12ff7-aab0-467d-ab00-68156601edf7-maas-api-service-ca\") pod \"5ab12ff7-aab0-467d-ab00-68156601edf7\" (UID: \"5ab12ff7-aab0-467d-ab00-68156601edf7\") "
Jul 17 11:43:06.379164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:06.379145    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ab12ff7-aab0-467d-ab00-68156601edf7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ab12ff7-aab0-467d-ab00-68156601edf7" (UID: "5ab12ff7-aab0-467d-ab00-68156601edf7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:43:06.480249 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:06.480171    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab12ff7-aab0-467d-ab00-68156601edf7-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:43:07.143110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:07.143073    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb" event={"ID":"5ab12ff7-aab0-467d-ab00-68156601edf7","Type":"ContainerDied","Data":"bc5b5c00756739b5caf3811ce0a11f6d8647cd6dcb4335b687208c9ea68a45e0"}
Jul 17 11:43:07.143110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:07.143095    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"
Jul 17 11:43:07.143341 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:07.143134    2566 scope.go:117] "RemoveContainer" containerID="228b49c1e5f1400c7854c14e5bd005812d3ec147567ec03f47ad093713ce88bf"
Jul 17 11:43:07.166050 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:07.166025    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"]
Jul 17 11:43:07.172100 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:07.172075    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l4kwb"]
Jul 17 11:43:07.658201 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:07.658162    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" path="/var/lib/kubelet/pods/5ab12ff7-aab0-467d-ab00-68156601edf7/volumes"
Jul 17 11:43:18.747662 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.747623    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"]
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.747993    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerName="revoke-keys"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748004    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerName="revoke-keys"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748016    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerName="revoke-keys"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748023    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerName="revoke-keys"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748034    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748040    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748053    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerName="revoke-keys"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748059    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerName="revoke-keys"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748064    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerName="revoke-keys"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748069    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerName="revoke-keys"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748085    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerName="revoke-keys"
Jul 17 11:43:18.748095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748091    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerName="revoke-keys"
Jul 17 11:43:18.748472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748142    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c74c80eb-4ad6-431c-b227-2669d494ec2b" containerName="istio-proxy"
Jul 17 11:43:18.748472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748149    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerName="revoke-keys"
Jul 17 11:43:18.748472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748156    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerName="revoke-keys"
Jul 17 11:43:18.748472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748162    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a0f2abd-6201-443a-a71c-68457975aca7" containerName="revoke-keys"
Jul 17 11:43:18.748472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748169    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerName="revoke-keys"
Jul 17 11:43:18.748472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748174    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerName="revoke-keys"
Jul 17 11:43:18.748472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.748180    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23f6ab4-8f76-44c1-9382-d677e49e6e5e" containerName="revoke-keys"
Jul 17 11:43:18.752286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.752269    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"
Jul 17 11:43:18.755665 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.755641    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:43:18.755834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.755645    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:43:18.761080 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.761055    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"]
Jul 17 11:43:18.891681 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.891644    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-n879q\" (UID: \"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"
Jul 17 11:43:18.992991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.992950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-n879q\" (UID: \"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"
Jul 17 11:43:18.993624 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:18.993599    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-n879q\" (UID: \"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"
Jul 17 11:43:19.063284 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:19.063211    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"
Jul 17 11:43:19.193306 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:19.193283    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"]
Jul 17 11:43:19.195267 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:43:19.195228    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f9d8de1_15a6_4e44_8b94_09a2fa60fb4a.slice/crio-5a14fd0b9933f006b5a2ceed7fe207e133c5017d091d5f1eaec7a4f6a8c18d97 WatchSource:0}: Error finding container 5a14fd0b9933f006b5a2ceed7fe207e133c5017d091d5f1eaec7a4f6a8c18d97: Status 404 returned error can't find the container with id 5a14fd0b9933f006b5a2ceed7fe207e133c5017d091d5f1eaec7a4f6a8c18d97
Jul 17 11:43:19.204306 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:19.204281    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" event={"ID":"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a","Type":"ContainerStarted","Data":"5a14fd0b9933f006b5a2ceed7fe207e133c5017d091d5f1eaec7a4f6a8c18d97"}
Jul 17 11:43:20.211052 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:20.210929    2566 generic.go:356] "Generic (PLEG): container finished" podID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerID="74155c3efa99549430be2808b57e6a1943639b8f28f0887787c59afa073b8298" exitCode=6
Jul 17 11:43:20.211052 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:20.211015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" event={"ID":"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a","Type":"ContainerDied","Data":"74155c3efa99549430be2808b57e6a1943639b8f28f0887787c59afa073b8298"}
Jul 17 11:43:20.211538 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:20.211196    2566 scope.go:117] "RemoveContainer" containerID="74155c3efa99549430be2808b57e6a1943639b8f28f0887787c59afa073b8298"
Jul 17 11:43:21.218057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:21.217976    2566 generic.go:356] "Generic (PLEG): container finished" podID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerID="8dd279be9f048b0195cb020c5f2ae8062b6fe3f8c629fddc85adb2ff93a2d971" exitCode=6
Jul 17 11:43:21.218437 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:21.218059    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" event={"ID":"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a","Type":"ContainerDied","Data":"8dd279be9f048b0195cb020c5f2ae8062b6fe3f8c629fddc85adb2ff93a2d971"}
Jul 17 11:43:21.218437 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:21.218118    2566 scope.go:117] "RemoveContainer" containerID="74155c3efa99549430be2808b57e6a1943639b8f28f0887787c59afa073b8298"
Jul 17 11:43:21.218437 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:21.218329    2566 scope.go:117] "RemoveContainer" containerID="8dd279be9f048b0195cb020c5f2ae8062b6fe3f8c629fddc85adb2ff93a2d971"
Jul 17 11:43:21.218580 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:21.218562    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-n879q_odh-ai-gateway-infra(2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a"
Jul 17 11:43:22.224898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:22.224872    2566 scope.go:117] "RemoveContainer" containerID="8dd279be9f048b0195cb020c5f2ae8062b6fe3f8c629fddc85adb2ff93a2d971"
Jul 17 11:43:22.225254 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:22.225064    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-n879q_odh-ai-gateway-infra(2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a"
Jul 17 11:43:25.507092 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.507060    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"]
Jul 17 11:43:25.507492 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.507477    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerName="revoke-keys"
Jul 17 11:43:25.507492 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.507493    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerName="revoke-keys"
Jul 17 11:43:25.507582 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.507565    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ab12ff7-aab0-467d-ab00-68156601edf7" containerName="revoke-keys"
Jul 17 11:43:25.511641 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.511625    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"
Jul 17 11:43:25.518935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.518803    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"]
Jul 17 11:43:25.653807 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.653775    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a22ad0-92ff-4f08-b88e-41596f69ed79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-gwprb\" (UID: \"39a22ad0-92ff-4f08-b88e-41596f69ed79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"
Jul 17 11:43:25.754821 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.754785    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a22ad0-92ff-4f08-b88e-41596f69ed79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-gwprb\" (UID: \"39a22ad0-92ff-4f08-b88e-41596f69ed79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"
Jul 17 11:43:25.755358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.755340    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a22ad0-92ff-4f08-b88e-41596f69ed79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-gwprb\" (UID: \"39a22ad0-92ff-4f08-b88e-41596f69ed79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"
Jul 17 11:43:25.823463 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:25.823437    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"
Jul 17 11:43:26.160427 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:26.160401    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"]
Jul 17 11:43:26.162166 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:43:26.162139    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a22ad0_92ff_4f08_b88e_41596f69ed79.slice/crio-09b98642b4ebc1460d6fb77c018cbdd7cdb7eee5821029d1d29e7baabce26a81 WatchSource:0}: Error finding container 09b98642b4ebc1460d6fb77c018cbdd7cdb7eee5821029d1d29e7baabce26a81: Status 404 returned error can't find the container with id 09b98642b4ebc1460d6fb77c018cbdd7cdb7eee5821029d1d29e7baabce26a81
Jul 17 11:43:26.244670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:26.244638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb" event={"ID":"39a22ad0-92ff-4f08-b88e-41596f69ed79","Type":"ContainerStarted","Data":"09b98642b4ebc1460d6fb77c018cbdd7cdb7eee5821029d1d29e7baabce26a81"}
Jul 17 11:43:27.250798 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.250765    2566 generic.go:356] "Generic (PLEG): container finished" podID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerID="8856c7bec0eb4abcc23eb6307441690294f9fbc0d08774b7ba03e67e14d7b7fb" exitCode=6
Jul 17 11:43:27.251178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.250848    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb" event={"ID":"39a22ad0-92ff-4f08-b88e-41596f69ed79","Type":"ContainerDied","Data":"8856c7bec0eb4abcc23eb6307441690294f9fbc0d08774b7ba03e67e14d7b7fb"}
Jul 17 11:43:27.251178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.251016    2566 scope.go:117] "RemoveContainer" containerID="8856c7bec0eb4abcc23eb6307441690294f9fbc0d08774b7ba03e67e14d7b7fb"
Jul 17 11:43:27.629405 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.629375    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"]
Jul 17 11:43:27.633311 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.633294    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"
Jul 17 11:43:27.639525 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.639499    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"]
Jul 17 11:43:27.775893 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.775849    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9af2-a239-43b1-9a31-49a385a3fa79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-6c9t2\" (UID: \"3d5e9af2-a239-43b1-9a31-49a385a3fa79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"
Jul 17 11:43:27.877572 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.877520    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9af2-a239-43b1-9a31-49a385a3fa79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-6c9t2\" (UID: \"3d5e9af2-a239-43b1-9a31-49a385a3fa79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"
Jul 17 11:43:27.878202 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.878182    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9af2-a239-43b1-9a31-49a385a3fa79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-6c9t2\" (UID: \"3d5e9af2-a239-43b1-9a31-49a385a3fa79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"
Jul 17 11:43:27.944766 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:27.944687    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"
Jul 17 11:43:28.069516 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:28.069492    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"]
Jul 17 11:43:28.071458 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:43:28.071435    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5e9af2_a239_43b1_9a31_49a385a3fa79.slice/crio-5221f76116d2a5c5224fb509a4fdedd727fadca522ed9e2e26967cd0320c244c WatchSource:0}: Error finding container 5221f76116d2a5c5224fb509a4fdedd727fadca522ed9e2e26967cd0320c244c: Status 404 returned error can't find the container with id 5221f76116d2a5c5224fb509a4fdedd727fadca522ed9e2e26967cd0320c244c
Jul 17 11:43:28.256897 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:28.256873    2566 generic.go:356] "Generic (PLEG): container finished" podID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerID="7d3a0d0037b59abc55199c4242aa8f2c700af81cdd060b668bf400b6a518d345" exitCode=6
Jul 17 11:43:28.257237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:28.256944    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb" event={"ID":"39a22ad0-92ff-4f08-b88e-41596f69ed79","Type":"ContainerDied","Data":"7d3a0d0037b59abc55199c4242aa8f2c700af81cdd060b668bf400b6a518d345"}
Jul 17 11:43:28.257237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:28.257009    2566 scope.go:117] "RemoveContainer" containerID="8856c7bec0eb4abcc23eb6307441690294f9fbc0d08774b7ba03e67e14d7b7fb"
Jul 17 11:43:28.257393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:28.257368    2566 scope.go:117] "RemoveContainer" containerID="7d3a0d0037b59abc55199c4242aa8f2c700af81cdd060b668bf400b6a518d345"
Jul 17 11:43:28.257662 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:28.257638    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-gwprb_odh-ai-gateway-infra(39a22ad0-92ff-4f08-b88e-41596f69ed79)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79"
Jul 17 11:43:28.258793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:28.258770    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" event={"ID":"3d5e9af2-a239-43b1-9a31-49a385a3fa79","Type":"ContainerStarted","Data":"5221f76116d2a5c5224fb509a4fdedd727fadca522ed9e2e26967cd0320c244c"}
Jul 17 11:43:29.265462 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:29.265434    2566 scope.go:117] "RemoveContainer" containerID="7d3a0d0037b59abc55199c4242aa8f2c700af81cdd060b668bf400b6a518d345"
Jul 17 11:43:29.265930 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:29.265646    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-gwprb_odh-ai-gateway-infra(39a22ad0-92ff-4f08-b88e-41596f69ed79)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79"
Jul 17 11:43:29.266738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:29.266700    2566 generic.go:356] "Generic (PLEG): container finished" podID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerID="3801396a8c59770e37e650ad9e27725bcf64041910486e2c9b7129acde6f9154" exitCode=6
Jul 17 11:43:29.266826 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:29.266803    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" event={"ID":"3d5e9af2-a239-43b1-9a31-49a385a3fa79","Type":"ContainerDied","Data":"3801396a8c59770e37e650ad9e27725bcf64041910486e2c9b7129acde6f9154"}
Jul 17 11:43:29.266941 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:29.266928    2566 scope.go:117] "RemoveContainer" containerID="3801396a8c59770e37e650ad9e27725bcf64041910486e2c9b7129acde6f9154"
Jul 17 11:43:30.273878 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:30.273791    2566 generic.go:356] "Generic (PLEG): container finished" podID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerID="b74934150b98b320040c1c5507a1b7cf9f1366fc4ee5575c25b898268210be06" exitCode=6
Jul 17 11:43:30.274247 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:30.273881    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" event={"ID":"3d5e9af2-a239-43b1-9a31-49a385a3fa79","Type":"ContainerDied","Data":"b74934150b98b320040c1c5507a1b7cf9f1366fc4ee5575c25b898268210be06"}
Jul 17 11:43:30.274247 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:30.273939    2566 scope.go:117] "RemoveContainer" containerID="3801396a8c59770e37e650ad9e27725bcf64041910486e2c9b7129acde6f9154"
Jul 17 11:43:30.274247 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:30.274158    2566 scope.go:117] "RemoveContainer" containerID="b74934150b98b320040c1c5507a1b7cf9f1366fc4ee5575c25b898268210be06"
Jul 17 11:43:30.274415 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:30.274396    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-6c9t2_odh-ai-gateway-infra(3d5e9af2-a239-43b1-9a31-49a385a3fa79)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79"
Jul 17 11:43:31.280776 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:31.280751    2566 scope.go:117] "RemoveContainer" containerID="b74934150b98b320040c1c5507a1b7cf9f1366fc4ee5575c25b898268210be06"
Jul 17 11:43:31.281169 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:31.280944    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-6c9t2_odh-ai-gateway-infra(3d5e9af2-a239-43b1-9a31-49a385a3fa79)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79"
Jul 17 11:43:34.569172 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.569141    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"]
Jul 17 11:43:34.573678 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.573657    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.576521 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.576498    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-fc3287-openshift-default-dockercfg-sbptg\""
Jul 17 11:43:34.587028 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.587004    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"]
Jul 17 11:43:34.628577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.628551    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-istio-token\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.628771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.628605    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-envoy\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.628771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.628641    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-data\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.628771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.628678    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08472687-6ac3-4870-b620-d1ffa890c449-istio-podinfo\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.628771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.628696    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08472687-6ac3-4870-b620-d1ffa890c449-istiod-ca-cert\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.628771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.628730    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr8z9\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-kube-api-access-kr8z9\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.628771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.628772    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-certs\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.629009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.628792    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-socket\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.629009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.628811    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-credential-socket\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.651959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.651931    2566 scope.go:117] "RemoveContainer" containerID="8dd279be9f048b0195cb020c5f2ae8062b6fe3f8c629fddc85adb2ff93a2d971"
Jul 17 11:43:34.730259 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-credential-socket\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.730457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730289    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-istio-token\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.730457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730379    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-envoy\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.730457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-data\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.730637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730461    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08472687-6ac3-4870-b620-d1ffa890c449-istio-podinfo\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.730637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730483    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08472687-6ac3-4870-b620-d1ffa890c449-istiod-ca-cert\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.730637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730509    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr8z9\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-kube-api-access-kr8z9\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.730637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730572    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-certs\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.730637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730622    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-socket\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.730928 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.730632    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-credential-socket\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.731157 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.731126    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-socket\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.731275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.731218    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08472687-6ac3-4870-b620-d1ffa890c449-istiod-ca-cert\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.731275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.731228    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-data\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.731397 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.731283    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-certs\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.733132 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.733106    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-envoy\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.733554 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.733534    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08472687-6ac3-4870-b620-d1ffa890c449-istio-podinfo\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.740039 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.740021    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-istio-token\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.740251 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.740234    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr8z9\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-kube-api-access-kr8z9\") pod \"e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") " pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:34.883281 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:34.883245    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:35.026067 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.026042    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"]
Jul 17 11:43:35.028808 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:43:35.028768    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08472687_6ac3_4870_b620_d1ffa890c449.slice/crio-6e921e2b6aa98fadbb862398d7eeb49a1ff574690a5c05c2691ffbb2a4622b69 WatchSource:0}: Error finding container 6e921e2b6aa98fadbb862398d7eeb49a1ff574690a5c05c2691ffbb2a4622b69: Status 404 returned error can't find the container with id 6e921e2b6aa98fadbb862398d7eeb49a1ff574690a5c05c2691ffbb2a4622b69
Jul 17 11:43:35.031170 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.031137    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:43:35.031304 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.031212    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:43:35.031304 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.031243    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:43:35.301895 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.301809    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" event={"ID":"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a","Type":"ContainerStarted","Data":"960444751adc1d954236097649541b6604e19bd56094f1268ae68b5988b3e18d"}
Jul 17 11:43:35.303393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.303367    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" event={"ID":"08472687-6ac3-4870-b620-d1ffa890c449","Type":"ContainerStarted","Data":"cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419"}
Jul 17 11:43:35.303552 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.303414    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" event={"ID":"08472687-6ac3-4870-b620-d1ffa890c449","Type":"ContainerStarted","Data":"6e921e2b6aa98fadbb862398d7eeb49a1ff574690a5c05c2691ffbb2a4622b69"}
Jul 17 11:43:35.336077 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.336020    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" podStartSLOduration=17.335999556 podStartE2EDuration="17.335999556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:43:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:43:35.333353057 +0000 UTC m=+2798.190437002" watchObservedRunningTime="2026-07-17 11:43:35.335999556 +0000 UTC m=+2798.193083501"
Jul 17 11:43:35.366787 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.366741    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" podStartSLOduration=1.3667278729999999 podStartE2EDuration="1.366727873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:43:35.365067971 +0000 UTC m=+2798.222151933" watchObservedRunningTime="2026-07-17 11:43:35.366727873 +0000 UTC m=+2798.223811812"
Jul 17 11:43:35.883791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:35.883762    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:36.309667 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:36.309628    2566 generic.go:356] "Generic (PLEG): container finished" podID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerID="960444751adc1d954236097649541b6604e19bd56094f1268ae68b5988b3e18d" exitCode=6
Jul 17 11:43:36.309859 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:36.309725    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" event={"ID":"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a","Type":"ContainerDied","Data":"960444751adc1d954236097649541b6604e19bd56094f1268ae68b5988b3e18d"}
Jul 17 11:43:36.309859 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:36.309813    2566 scope.go:117] "RemoveContainer" containerID="8dd279be9f048b0195cb020c5f2ae8062b6fe3f8c629fddc85adb2ff93a2d971"
Jul 17 11:43:36.310199 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:36.310175    2566 scope.go:117] "RemoveContainer" containerID="960444751adc1d954236097649541b6604e19bd56094f1268ae68b5988b3e18d"
Jul 17 11:43:36.310401 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:36.310373    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-n879q_odh-ai-gateway-infra(2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a"
Jul 17 11:43:36.344524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:36.344492    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"]
Jul 17 11:43:36.888372 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:36.888337    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:37.316919 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:37.316538    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:37.317398 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:37.317379    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:37.463907 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:37.463883    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"
Jul 17 11:43:37.560872 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:37.560841    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a-maas-api-service-ca\") pod \"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a\" (UID: \"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a\") "
Jul 17 11:43:37.561216 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:37.561189    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" (UID: "2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:43:37.661414 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:37.661381    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:43:38.322778 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:38.322748    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"
Jul 17 11:43:38.322778 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:38.322769    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q" event={"ID":"2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a","Type":"ContainerDied","Data":"5a14fd0b9933f006b5a2ceed7fe207e133c5017d091d5f1eaec7a4f6a8c18d97"}
Jul 17 11:43:38.323298 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:38.322835    2566 scope.go:117] "RemoveContainer" containerID="960444751adc1d954236097649541b6604e19bd56094f1268ae68b5988b3e18d"
Jul 17 11:43:38.349333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:38.349310    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"]
Jul 17 11:43:38.356445 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:38.356425    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-n879q"]
Jul 17 11:43:39.651492 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:39.651464    2566 scope.go:117] "RemoveContainer" containerID="7d3a0d0037b59abc55199c4242aa8f2c700af81cdd060b668bf400b6a518d345"
Jul 17 11:43:39.657725 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:39.657683    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" path="/var/lib/kubelet/pods/2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a/volumes"
Jul 17 11:43:40.336334 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:40.336301    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb" event={"ID":"39a22ad0-92ff-4f08-b88e-41596f69ed79","Type":"ContainerStarted","Data":"e9126100da17a099f6d750ac3cdbbbd343bcea47bddbc8feb16e1d06a66e9375"}
Jul 17 11:43:41.344145 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:41.344103    2566 generic.go:356] "Generic (PLEG): container finished" podID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerID="e9126100da17a099f6d750ac3cdbbbd343bcea47bddbc8feb16e1d06a66e9375" exitCode=6
Jul 17 11:43:41.344631 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:41.344179    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb" event={"ID":"39a22ad0-92ff-4f08-b88e-41596f69ed79","Type":"ContainerDied","Data":"e9126100da17a099f6d750ac3cdbbbd343bcea47bddbc8feb16e1d06a66e9375"}
Jul 17 11:43:41.344631 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:41.344295    2566 scope.go:117] "RemoveContainer" containerID="7d3a0d0037b59abc55199c4242aa8f2c700af81cdd060b668bf400b6a518d345"
Jul 17 11:43:41.344631 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:41.344564    2566 scope.go:117] "RemoveContainer" containerID="e9126100da17a099f6d750ac3cdbbbd343bcea47bddbc8feb16e1d06a66e9375"
Jul 17 11:43:41.344906 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:41.344884    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-gwprb_odh-ai-gateway-infra(39a22ad0-92ff-4f08-b88e-41596f69ed79)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79"
Jul 17 11:43:41.371724 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:41.371684    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"]
Jul 17 11:43:42.483906 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:42.483883    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"
Jul 17 11:43:42.503951 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:42.503916    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a22ad0-92ff-4f08-b88e-41596f69ed79-maas-api-service-ca\") pod \"39a22ad0-92ff-4f08-b88e-41596f69ed79\" (UID: \"39a22ad0-92ff-4f08-b88e-41596f69ed79\") "
Jul 17 11:43:42.504370 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:42.504342    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39a22ad0-92ff-4f08-b88e-41596f69ed79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "39a22ad0-92ff-4f08-b88e-41596f69ed79" (UID: "39a22ad0-92ff-4f08-b88e-41596f69ed79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:43:42.605417 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:42.605339    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a22ad0-92ff-4f08-b88e-41596f69ed79-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:43:43.357489 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:43.357458    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"
Jul 17 11:43:43.357489 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:43.357470    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb" event={"ID":"39a22ad0-92ff-4f08-b88e-41596f69ed79","Type":"ContainerDied","Data":"09b98642b4ebc1460d6fb77c018cbdd7cdb7eee5821029d1d29e7baabce26a81"}
Jul 17 11:43:43.357747 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:43.357526    2566 scope.go:117] "RemoveContainer" containerID="e9126100da17a099f6d750ac3cdbbbd343bcea47bddbc8feb16e1d06a66e9375"
Jul 17 11:43:43.384014 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:43.383992    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"]
Jul 17 11:43:43.386599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:43.386579    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-gwprb"]
Jul 17 11:43:43.657944 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:43.657871    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" path="/var/lib/kubelet/pods/39a22ad0-92ff-4f08-b88e-41596f69ed79/volumes"
Jul 17 11:43:45.151259 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:45.151224    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:43:45.652012 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:45.651986    2566 scope.go:117] "RemoveContainer" containerID="b74934150b98b320040c1c5507a1b7cf9f1366fc4ee5575c25b898268210be06"
Jul 17 11:43:46.375893 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:46.375859    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" event={"ID":"3d5e9af2-a239-43b1-9a31-49a385a3fa79","Type":"ContainerStarted","Data":"49d1e56159c67d5fb8ae92dcb9b1deb04e8ca52683f49723895c76ae6303eebc"}
Jul 17 11:43:46.404916 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:46.404868    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" podStartSLOduration=19.404853743 podStartE2EDuration="19.404853743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:43:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:43:46.404336455 +0000 UTC m=+2809.261420401" watchObservedRunningTime="2026-07-17 11:43:46.404853743 +0000 UTC m=+2809.261937687"
Jul 17 11:43:47.383355 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:47.383323    2566 generic.go:356] "Generic (PLEG): container finished" podID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerID="49d1e56159c67d5fb8ae92dcb9b1deb04e8ca52683f49723895c76ae6303eebc" exitCode=6
Jul 17 11:43:47.383773 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:47.383405    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" event={"ID":"3d5e9af2-a239-43b1-9a31-49a385a3fa79","Type":"ContainerDied","Data":"49d1e56159c67d5fb8ae92dcb9b1deb04e8ca52683f49723895c76ae6303eebc"}
Jul 17 11:43:47.383773 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:47.383468    2566 scope.go:117] "RemoveContainer" containerID="b74934150b98b320040c1c5507a1b7cf9f1366fc4ee5575c25b898268210be06"
Jul 17 11:43:47.383889 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:47.383825    2566 scope.go:117] "RemoveContainer" containerID="49d1e56159c67d5fb8ae92dcb9b1deb04e8ca52683f49723895c76ae6303eebc"
Jul 17 11:43:47.384028 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:43:47.384005    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-6c9t2_odh-ai-gateway-infra(3d5e9af2-a239-43b1-9a31-49a385a3fa79)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79"
Jul 17 11:43:47.422110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:47.422083    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"]
Jul 17 11:43:48.575577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:48.575555    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"
Jul 17 11:43:48.664111 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:48.664033    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9af2-a239-43b1-9a31-49a385a3fa79-maas-api-service-ca\") pod \"3d5e9af2-a239-43b1-9a31-49a385a3fa79\" (UID: \"3d5e9af2-a239-43b1-9a31-49a385a3fa79\") "
Jul 17 11:43:48.664351 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:48.664329    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d5e9af2-a239-43b1-9a31-49a385a3fa79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3d5e9af2-a239-43b1-9a31-49a385a3fa79" (UID: "3d5e9af2-a239-43b1-9a31-49a385a3fa79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:43:48.765577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:48.765538    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9af2-a239-43b1-9a31-49a385a3fa79-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:43:49.397139 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:49.397113    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"
Jul 17 11:43:49.397357 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:49.397138    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2" event={"ID":"3d5e9af2-a239-43b1-9a31-49a385a3fa79","Type":"ContainerDied","Data":"5221f76116d2a5c5224fb509a4fdedd727fadca522ed9e2e26967cd0320c244c"}
Jul 17 11:43:49.397357 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:49.397199    2566 scope.go:117] "RemoveContainer" containerID="49d1e56159c67d5fb8ae92dcb9b1deb04e8ca52683f49723895c76ae6303eebc"
Jul 17 11:43:49.421503 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:49.421481    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"]
Jul 17 11:43:49.426014 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:49.425993    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-6c9t2"]
Jul 17 11:43:49.659347 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:49.659267    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" path="/var/lib/kubelet/pods/3d5e9af2-a239-43b1-9a31-49a385a3fa79/volumes"
Jul 17 11:43:53.172627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:53.171505    2566 patch_prober.go:28] interesting pod/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": read tcp 10.132.0.2:57224->10.132.0.96:15021: read: connection reset by peer" start-of-body=
Jul 17 11:43:53.172627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:53.171579    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": read tcp 10.132.0.2:57224->10.132.0.96:15021: read: connection reset by peer"
Jul 17 11:43:53.421431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:53.421336    2566 generic.go:356] "Generic (PLEG): container finished" podID="08472687-6ac3-4870-b620-d1ffa890c449" containerID="cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419" exitCode=0
Jul 17 11:43:53.421431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:53.421376    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" event={"ID":"08472687-6ac3-4870-b620-d1ffa890c449","Type":"ContainerDied","Data":"cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419"}
Jul 17 11:43:53.421788 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:53.421763    2566 scope.go:117] "RemoveContainer" containerID="cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419"
Jul 17 11:43:53.422764 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:53.422734    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:43:53.422862 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:53.422799    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:43:53.422862 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:53.422828    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:43:54.428405 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:54.428369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" event={"ID":"08472687-6ac3-4870-b620-d1ffa890c449","Type":"ContainerStarted","Data":"e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e"}
Jul 17 11:43:54.883861 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:54.883825    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:54.884096 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:54.884070    2566 patch_prober.go:28] interesting pod/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Jul 17 11:43:54.884188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:54.884128    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Jul 17 11:43:55.883933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:55.883895    2566 patch_prober.go:28] interesting pod/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Jul 17 11:43:55.884323 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:55.883953    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Jul 17 11:43:56.884238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:56.884199    2566 patch_prober.go:28] interesting pod/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Jul 17 11:43:56.884738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:56.884265    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Jul 17 11:43:57.883553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:57.883521    2566 patch_prober.go:28] interesting pod/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Jul 17 11:43:57.883738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:57.883574    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Jul 17 11:43:58.247080 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:58.247043    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:43:58.887261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:58.887230    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:58.887621 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:58.887599    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:43:58.888309 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:43:58.888289    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:44:08.818184 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818151    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"]
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818544    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818555    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818572    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818577    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818587    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818593    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818600    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818606    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818612    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818618    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818625    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818631    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818639    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerName="revoke-keys"
Jul 17 11:44:08.818654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818645    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerName="revoke-keys"
Jul 17 11:44:08.819120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818719    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerName="revoke-keys"
Jul 17 11:44:08.819120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818728    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerName="revoke-keys"
Jul 17 11:44:08.819120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818736    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerName="revoke-keys"
Jul 17 11:44:08.819120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818741    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerName="revoke-keys"
Jul 17 11:44:08.819120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818747    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerName="revoke-keys"
Jul 17 11:44:08.819120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818752    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerName="revoke-keys"
Jul 17 11:44:08.819120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.818757    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerName="revoke-keys"
Jul 17 11:44:08.821649 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.821632    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"
Jul 17 11:44:08.824948 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.824920    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:44:08.826833 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.826815    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:44:08.833537 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.833513    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"]
Jul 17 11:44:08.963541 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:08.963507    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2209d0fa-cd46-4a87-9a50-43cd31411ce0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl\" (UID: \"2209d0fa-cd46-4a87-9a50-43cd31411ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"
Jul 17 11:44:09.064900 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:09.064855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2209d0fa-cd46-4a87-9a50-43cd31411ce0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl\" (UID: \"2209d0fa-cd46-4a87-9a50-43cd31411ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"
Jul 17 11:44:09.065490 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:09.065470    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2209d0fa-cd46-4a87-9a50-43cd31411ce0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl\" (UID: \"2209d0fa-cd46-4a87-9a50-43cd31411ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"
Jul 17 11:44:09.131762 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:09.131732    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"
Jul 17 11:44:09.288328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:09.288293    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"]
Jul 17 11:44:09.301013 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:44:09.300987    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2209d0fa_cd46_4a87_9a50_43cd31411ce0.slice/crio-41fb6a69132dbecea86b732083817cc9c06d068fd94bd9d3b3764b08bae47134 WatchSource:0}: Error finding container 41fb6a69132dbecea86b732083817cc9c06d068fd94bd9d3b3764b08bae47134: Status 404 returned error can't find the container with id 41fb6a69132dbecea86b732083817cc9c06d068fd94bd9d3b3764b08bae47134
Jul 17 11:44:09.509832 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:09.509740    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl" event={"ID":"2209d0fa-cd46-4a87-9a50-43cd31411ce0","Type":"ContainerStarted","Data":"41fb6a69132dbecea86b732083817cc9c06d068fd94bd9d3b3764b08bae47134"}
Jul 17 11:44:10.516009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:10.515976    2566 generic.go:356] "Generic (PLEG): container finished" podID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerID="cbc9a27978a90cbfc9e3dec3558300903ab86c4e221cfcbb0e5be1572fb55160" exitCode=6
Jul 17 11:44:10.516465 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:10.516056    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl" event={"ID":"2209d0fa-cd46-4a87-9a50-43cd31411ce0","Type":"ContainerDied","Data":"cbc9a27978a90cbfc9e3dec3558300903ab86c4e221cfcbb0e5be1572fb55160"}
Jul 17 11:44:10.516465 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:10.516226    2566 scope.go:117] "RemoveContainer" containerID="cbc9a27978a90cbfc9e3dec3558300903ab86c4e221cfcbb0e5be1572fb55160"
Jul 17 11:44:11.522556 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:11.522469    2566 generic.go:356] "Generic (PLEG): container finished" podID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerID="d2cad519208c7d43ca5ecdf27733dd6e0ae22df6acea0bf08d4c47f335bc2f97" exitCode=6
Jul 17 11:44:11.522992 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:11.522563    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl" event={"ID":"2209d0fa-cd46-4a87-9a50-43cd31411ce0","Type":"ContainerDied","Data":"d2cad519208c7d43ca5ecdf27733dd6e0ae22df6acea0bf08d4c47f335bc2f97"}
Jul 17 11:44:11.522992 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:11.522616    2566 scope.go:117] "RemoveContainer" containerID="cbc9a27978a90cbfc9e3dec3558300903ab86c4e221cfcbb0e5be1572fb55160"
Jul 17 11:44:11.522992 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:11.522758    2566 scope.go:117] "RemoveContainer" containerID="d2cad519208c7d43ca5ecdf27733dd6e0ae22df6acea0bf08d4c47f335bc2f97"
Jul 17 11:44:11.522992 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:11.522980    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl_odh-ai-gateway-infra(2209d0fa-cd46-4a87-9a50-43cd31411ce0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0"
Jul 17 11:44:12.528556 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:12.528529    2566 scope.go:117] "RemoveContainer" containerID="d2cad519208c7d43ca5ecdf27733dd6e0ae22df6acea0bf08d4c47f335bc2f97"
Jul 17 11:44:12.528954 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:12.528800    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl_odh-ai-gateway-infra(2209d0fa-cd46-4a87-9a50-43cd31411ce0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0"
Jul 17 11:44:14.046680 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:14.046643    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:44:15.565455 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.565420    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"]
Jul 17 11:44:15.565867 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.565834    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerName="revoke-keys"
Jul 17 11:44:15.565867 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.565849    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a22ad0-92ff-4f08-b88e-41596f69ed79" containerName="revoke-keys"
Jul 17 11:44:15.565939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.565886    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerName="revoke-keys"
Jul 17 11:44:15.565939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.565892    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerName="revoke-keys"
Jul 17 11:44:15.565998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.565954    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f9d8de1-15a6-4e44-8b94-09a2fa60fb4a" containerName="revoke-keys"
Jul 17 11:44:15.565998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.565962    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5e9af2-a239-43b1-9a31-49a385a3fa79" containerName="revoke-keys"
Jul 17 11:44:15.570254 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.570237    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"
Jul 17 11:44:15.579583 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.579556    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"]
Jul 17 11:44:15.612400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.612357    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ed96aa-77fd-4b21-aac3-77429e5ebd0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-6p7t4\" (UID: \"33ed96aa-77fd-4b21-aac3-77429e5ebd0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"
Jul 17 11:44:15.713257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.713200    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ed96aa-77fd-4b21-aac3-77429e5ebd0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-6p7t4\" (UID: \"33ed96aa-77fd-4b21-aac3-77429e5ebd0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"
Jul 17 11:44:15.713875 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.713855    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ed96aa-77fd-4b21-aac3-77429e5ebd0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-6p7t4\" (UID: \"33ed96aa-77fd-4b21-aac3-77429e5ebd0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"
Jul 17 11:44:15.882175 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:15.882137    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"
Jul 17 11:44:16.222652 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:16.222624    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"]
Jul 17 11:44:16.224501 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:44:16.224477    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ed96aa_77fd_4b21_aac3_77429e5ebd0d.slice/crio-de5fe488f9b9acdf004dfc9ad4582390c1605515ff2e995ebb17bf4cb0db331f WatchSource:0}: Error finding container de5fe488f9b9acdf004dfc9ad4582390c1605515ff2e995ebb17bf4cb0db331f: Status 404 returned error can't find the container with id de5fe488f9b9acdf004dfc9ad4582390c1605515ff2e995ebb17bf4cb0db331f
Jul 17 11:44:16.558700 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:16.558664    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" event={"ID":"33ed96aa-77fd-4b21-aac3-77429e5ebd0d","Type":"ContainerStarted","Data":"b45888fc2d9034ea10709f12eaac7ff910b0831f993224155e834608ebc29812"}
Jul 17 11:44:16.558878 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:16.558726    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" event={"ID":"33ed96aa-77fd-4b21-aac3-77429e5ebd0d","Type":"ContainerStarted","Data":"de5fe488f9b9acdf004dfc9ad4582390c1605515ff2e995ebb17bf4cb0db331f"}
Jul 17 11:44:16.594381 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:16.594329    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" podStartSLOduration=1.5943151169999998 podStartE2EDuration="1.594315117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:44:16.593901666 +0000 UTC m=+2839.450985610" watchObservedRunningTime="2026-07-17 11:44:16.594315117 +0000 UTC m=+2839.451399059"
Jul 17 11:44:17.565286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:17.565243    2566 generic.go:356] "Generic (PLEG): container finished" podID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerID="b45888fc2d9034ea10709f12eaac7ff910b0831f993224155e834608ebc29812" exitCode=6
Jul 17 11:44:17.565286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:17.565288    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" event={"ID":"33ed96aa-77fd-4b21-aac3-77429e5ebd0d","Type":"ContainerDied","Data":"b45888fc2d9034ea10709f12eaac7ff910b0831f993224155e834608ebc29812"}
Jul 17 11:44:17.565529 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:17.565511    2566 scope.go:117] "RemoveContainer" containerID="b45888fc2d9034ea10709f12eaac7ff910b0831f993224155e834608ebc29812"
Jul 17 11:44:18.571451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:18.571367    2566 generic.go:356] "Generic (PLEG): container finished" podID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerID="510f8a78c8721639803edd5f3030eb5a904be20c1283c161064b882ba15ce690" exitCode=6
Jul 17 11:44:18.571451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:18.571416    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" event={"ID":"33ed96aa-77fd-4b21-aac3-77429e5ebd0d","Type":"ContainerDied","Data":"510f8a78c8721639803edd5f3030eb5a904be20c1283c161064b882ba15ce690"}
Jul 17 11:44:18.571939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:18.571466    2566 scope.go:117] "RemoveContainer" containerID="b45888fc2d9034ea10709f12eaac7ff910b0831f993224155e834608ebc29812"
Jul 17 11:44:18.571939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:18.571612    2566 scope.go:117] "RemoveContainer" containerID="510f8a78c8721639803edd5f3030eb5a904be20c1283c161064b882ba15ce690"
Jul 17 11:44:18.571939 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:18.571839    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-6p7t4_odh-ai-gateway-infra(33ed96aa-77fd-4b21-aac3-77429e5ebd0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d"
Jul 17 11:44:19.578905 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:19.578880    2566 scope.go:117] "RemoveContainer" containerID="510f8a78c8721639803edd5f3030eb5a904be20c1283c161064b882ba15ce690"
Jul 17 11:44:19.579323 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:19.579048    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-6p7t4_odh-ai-gateway-infra(33ed96aa-77fd-4b21-aac3-77429e5ebd0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d"
Jul 17 11:44:23.651892 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:23.651861    2566 scope.go:117] "RemoveContainer" containerID="d2cad519208c7d43ca5ecdf27733dd6e0ae22df6acea0bf08d4c47f335bc2f97"
Jul 17 11:44:24.604036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:24.603948    2566 generic.go:356] "Generic (PLEG): container finished" podID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerID="26b639d34b249da7f543867e6981344208129e981b0651d3d837093e1cf9e23f" exitCode=6
Jul 17 11:44:24.604036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:24.604017    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl" event={"ID":"2209d0fa-cd46-4a87-9a50-43cd31411ce0","Type":"ContainerDied","Data":"26b639d34b249da7f543867e6981344208129e981b0651d3d837093e1cf9e23f"}
Jul 17 11:44:24.604283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:24.604081    2566 scope.go:117] "RemoveContainer" containerID="d2cad519208c7d43ca5ecdf27733dd6e0ae22df6acea0bf08d4c47f335bc2f97"
Jul 17 11:44:24.604427 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:24.604411    2566 scope.go:117] "RemoveContainer" containerID="26b639d34b249da7f543867e6981344208129e981b0651d3d837093e1cf9e23f"
Jul 17 11:44:24.604666 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:24.604645    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl_odh-ai-gateway-infra(2209d0fa-cd46-4a87-9a50-43cd31411ce0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0"
Jul 17 11:44:25.633972 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:25.633938    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"]
Jul 17 11:44:25.767218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:25.767193    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"
Jul 17 11:44:25.802599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:25.802573    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2209d0fa-cd46-4a87-9a50-43cd31411ce0-maas-api-service-ca\") pod \"2209d0fa-cd46-4a87-9a50-43cd31411ce0\" (UID: \"2209d0fa-cd46-4a87-9a50-43cd31411ce0\") "
Jul 17 11:44:25.803009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:25.802980    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2209d0fa-cd46-4a87-9a50-43cd31411ce0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2209d0fa-cd46-4a87-9a50-43cd31411ce0" (UID: "2209d0fa-cd46-4a87-9a50-43cd31411ce0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:44:25.903837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:25.903759    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2209d0fa-cd46-4a87-9a50-43cd31411ce0-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:44:26.617237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:26.617214    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"
Jul 17 11:44:26.617415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:26.617207    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl" event={"ID":"2209d0fa-cd46-4a87-9a50-43cd31411ce0","Type":"ContainerDied","Data":"41fb6a69132dbecea86b732083817cc9c06d068fd94bd9d3b3764b08bae47134"}
Jul 17 11:44:26.617415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:26.617383    2566 scope.go:117] "RemoveContainer" containerID="26b639d34b249da7f543867e6981344208129e981b0651d3d837093e1cf9e23f"
Jul 17 11:44:26.643455 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:26.643428    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"]
Jul 17 11:44:26.649315 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:26.649293    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2w5pl"]
Jul 17 11:44:27.658530 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.658497    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" path="/var/lib/kubelet/pods/2209d0fa-cd46-4a87-9a50-43cd31411ce0/volumes"
Jul 17 11:44:27.846261 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.846230    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"]
Jul 17 11:44:27.846589 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.846577    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerName="revoke-keys"
Jul 17 11:44:27.846632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.846591    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerName="revoke-keys"
Jul 17 11:44:27.846632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.846605    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerName="revoke-keys"
Jul 17 11:44:27.846632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.846611    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerName="revoke-keys"
Jul 17 11:44:27.846632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.846620    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerName="revoke-keys"
Jul 17 11:44:27.846632 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.846625    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerName="revoke-keys"
Jul 17 11:44:27.846815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.846687    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerName="revoke-keys"
Jul 17 11:44:27.846815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.846696    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerName="revoke-keys"
Jul 17 11:44:27.851101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.851084    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"
Jul 17 11:44:27.857550 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.857523    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"]
Jul 17 11:44:27.919915 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:27.919823    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa75b23a-ed6b-4a75-a123-8c8000524d1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-zrstp\" (UID: \"aa75b23a-ed6b-4a75-a123-8c8000524d1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"
Jul 17 11:44:28.021346 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.021311    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa75b23a-ed6b-4a75-a123-8c8000524d1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-zrstp\" (UID: \"aa75b23a-ed6b-4a75-a123-8c8000524d1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"
Jul 17 11:44:28.021933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.021912    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa75b23a-ed6b-4a75-a123-8c8000524d1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-zrstp\" (UID: \"aa75b23a-ed6b-4a75-a123-8c8000524d1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"
Jul 17 11:44:28.162969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.162932    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"
Jul 17 11:44:28.293718 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.293676    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"]
Jul 17 11:44:28.295678 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:44:28.295650    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa75b23a_ed6b_4a75_a123_8c8000524d1a.slice/crio-8ab10bf242ea45f248515e40466af1aebe982f3cbf2d658df8cb57b8ed2b9b9d WatchSource:0}: Error finding container 8ab10bf242ea45f248515e40466af1aebe982f3cbf2d658df8cb57b8ed2b9b9d: Status 404 returned error can't find the container with id 8ab10bf242ea45f248515e40466af1aebe982f3cbf2d658df8cb57b8ed2b9b9d
Jul 17 11:44:28.629339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.629309    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" event={"ID":"aa75b23a-ed6b-4a75-a123-8c8000524d1a","Type":"ContainerStarted","Data":"bd8d0bb4b88258be23d0bbef37cd0b114dcd0396eff34627a7a9ee2c64b92cc4"}
Jul 17 11:44:28.629523 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.629363    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" event={"ID":"aa75b23a-ed6b-4a75-a123-8c8000524d1a","Type":"ContainerStarted","Data":"8ab10bf242ea45f248515e40466af1aebe982f3cbf2d658df8cb57b8ed2b9b9d"}
Jul 17 11:44:28.651194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.651147    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" podStartSLOduration=1.651133212 podStartE2EDuration="1.651133212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:44:28.648666664 +0000 UTC m=+2851.505750622" watchObservedRunningTime="2026-07-17 11:44:28.651133212 +0000 UTC m=+2851.508217156"
Jul 17 11:44:28.758879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.758804    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:44:28.914356 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.914331    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"]
Jul 17 11:44:28.914846 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.914827    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2209d0fa-cd46-4a87-9a50-43cd31411ce0" containerName="revoke-keys"
Jul 17 11:44:28.917854 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.917840    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"
Jul 17 11:44:28.927635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:28.927222    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"]
Jul 17 11:44:29.031970 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:29.031892    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04731422-78fc-4fbd-a19b-40140651f4e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw\" (UID: \"04731422-78fc-4fbd-a19b-40140651f4e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"
Jul 17 11:44:29.132933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:29.132907    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04731422-78fc-4fbd-a19b-40140651f4e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw\" (UID: \"04731422-78fc-4fbd-a19b-40140651f4e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"
Jul 17 11:44:29.133513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:29.133491    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04731422-78fc-4fbd-a19b-40140651f4e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw\" (UID: \"04731422-78fc-4fbd-a19b-40140651f4e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"
Jul 17 11:44:29.229260 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:29.229227    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"
Jul 17 11:44:29.361917 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:29.361892    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"]
Jul 17 11:44:29.363771 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:44:29.363746    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04731422_78fc_4fbd_a19b_40140651f4e1.slice/crio-491839590d1e4203fbbdbefb7a8f72367d66e79ecf64d355705589788852a19b WatchSource:0}: Error finding container 491839590d1e4203fbbdbefb7a8f72367d66e79ecf64d355705589788852a19b: Status 404 returned error can't find the container with id 491839590d1e4203fbbdbefb7a8f72367d66e79ecf64d355705589788852a19b
Jul 17 11:44:29.636474 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:29.636448    2566 generic.go:356] "Generic (PLEG): container finished" podID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerID="bd8d0bb4b88258be23d0bbef37cd0b114dcd0396eff34627a7a9ee2c64b92cc4" exitCode=6
Jul 17 11:44:29.636648 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:29.636533    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" event={"ID":"aa75b23a-ed6b-4a75-a123-8c8000524d1a","Type":"ContainerDied","Data":"bd8d0bb4b88258be23d0bbef37cd0b114dcd0396eff34627a7a9ee2c64b92cc4"}
Jul 17 11:44:29.636778 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:29.636763    2566 scope.go:117] "RemoveContainer" containerID="bd8d0bb4b88258be23d0bbef37cd0b114dcd0396eff34627a7a9ee2c64b92cc4"
Jul 17 11:44:29.637917 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:29.637894    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw" event={"ID":"04731422-78fc-4fbd-a19b-40140651f4e1","Type":"ContainerStarted","Data":"491839590d1e4203fbbdbefb7a8f72367d66e79ecf64d355705589788852a19b"}
Jul 17 11:44:30.644613 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:30.644567    2566 generic.go:356] "Generic (PLEG): container finished" podID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerID="9b7f41bed7b917bed12213c00ce09e2b28183ea21631dff5722f25c02d30bda8" exitCode=6
Jul 17 11:44:30.645075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:30.644649    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" event={"ID":"aa75b23a-ed6b-4a75-a123-8c8000524d1a","Type":"ContainerDied","Data":"9b7f41bed7b917bed12213c00ce09e2b28183ea21631dff5722f25c02d30bda8"}
Jul 17 11:44:30.645075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:30.644722    2566 scope.go:117] "RemoveContainer" containerID="bd8d0bb4b88258be23d0bbef37cd0b114dcd0396eff34627a7a9ee2c64b92cc4"
Jul 17 11:44:30.645075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:30.644920    2566 scope.go:117] "RemoveContainer" containerID="9b7f41bed7b917bed12213c00ce09e2b28183ea21631dff5722f25c02d30bda8"
Jul 17 11:44:30.645258 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:30.645115    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-zrstp_odh-ai-gateway-infra(aa75b23a-ed6b-4a75-a123-8c8000524d1a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a"
Jul 17 11:44:30.646339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:30.646318    2566 generic.go:356] "Generic (PLEG): container finished" podID="04731422-78fc-4fbd-a19b-40140651f4e1" containerID="25c8efc0568d30ba8a87d1be5a4ebbc05f21fc7d3d065b743d8d8ba4d761b8fc" exitCode=6
Jul 17 11:44:30.646436 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:30.646396    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw" event={"ID":"04731422-78fc-4fbd-a19b-40140651f4e1","Type":"ContainerDied","Data":"25c8efc0568d30ba8a87d1be5a4ebbc05f21fc7d3d065b743d8d8ba4d761b8fc"}
Jul 17 11:44:30.646555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:30.646515    2566 scope.go:117] "RemoveContainer" containerID="25c8efc0568d30ba8a87d1be5a4ebbc05f21fc7d3d065b743d8d8ba4d761b8fc"
Jul 17 11:44:31.652567 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:31.652539    2566 scope.go:117] "RemoveContainer" containerID="510f8a78c8721639803edd5f3030eb5a904be20c1283c161064b882ba15ce690"
Jul 17 11:44:31.654269 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:31.654248    2566 scope.go:117] "RemoveContainer" containerID="9b7f41bed7b917bed12213c00ce09e2b28183ea21631dff5722f25c02d30bda8"
Jul 17 11:44:31.654477 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:31.654454    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-zrstp_odh-ai-gateway-infra(aa75b23a-ed6b-4a75-a123-8c8000524d1a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a"
Jul 17 11:44:31.659032 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:31.659012    2566 generic.go:356] "Generic (PLEG): container finished" podID="04731422-78fc-4fbd-a19b-40140651f4e1" containerID="3c3bc646d5fdf69ec1e80c5ed73e1659c2997511b9281c8fbc7a3c5f9b41a9b0" exitCode=6
Jul 17 11:44:31.659246 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:31.659232    2566 scope.go:117] "RemoveContainer" containerID="3c3bc646d5fdf69ec1e80c5ed73e1659c2997511b9281c8fbc7a3c5f9b41a9b0"
Jul 17 11:44:31.659399 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:31.659382    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw_odh-ai-gateway-infra(04731422-78fc-4fbd-a19b-40140651f4e1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw" podUID="04731422-78fc-4fbd-a19b-40140651f4e1"
Jul 17 11:44:31.661346 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:31.661324    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw" event={"ID":"04731422-78fc-4fbd-a19b-40140651f4e1","Type":"ContainerDied","Data":"3c3bc646d5fdf69ec1e80c5ed73e1659c2997511b9281c8fbc7a3c5f9b41a9b0"}
Jul 17 11:44:31.661453 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:31.661371    2566 scope.go:117] "RemoveContainer" containerID="25c8efc0568d30ba8a87d1be5a4ebbc05f21fc7d3d065b743d8d8ba4d761b8fc"
Jul 17 11:44:32.665801 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:32.665772    2566 generic.go:356] "Generic (PLEG): container finished" podID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerID="5b20a9cce4e5cc6816002a1a2325f7ad3d557c9a2c33379f617997881ab3604d" exitCode=6
Jul 17 11:44:32.666169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:32.665847    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" event={"ID":"33ed96aa-77fd-4b21-aac3-77429e5ebd0d","Type":"ContainerDied","Data":"5b20a9cce4e5cc6816002a1a2325f7ad3d557c9a2c33379f617997881ab3604d"}
Jul 17 11:44:32.666169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:32.665920    2566 scope.go:117] "RemoveContainer" containerID="510f8a78c8721639803edd5f3030eb5a904be20c1283c161064b882ba15ce690"
Jul 17 11:44:32.666300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:32.666210    2566 scope.go:117] "RemoveContainer" containerID="5b20a9cce4e5cc6816002a1a2325f7ad3d557c9a2c33379f617997881ab3604d"
Jul 17 11:44:32.666473 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:32.666452    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-6p7t4_odh-ai-gateway-infra(33ed96aa-77fd-4b21-aac3-77429e5ebd0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d"
Jul 17 11:44:32.667583 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:32.667569    2566 scope.go:117] "RemoveContainer" containerID="3c3bc646d5fdf69ec1e80c5ed73e1659c2997511b9281c8fbc7a3c5f9b41a9b0"
Jul 17 11:44:32.667745 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:32.667728    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw_odh-ai-gateway-infra(04731422-78fc-4fbd-a19b-40140651f4e1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw" podUID="04731422-78fc-4fbd-a19b-40140651f4e1"
Jul 17 11:44:33.698940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:33.698912    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"]
Jul 17 11:44:33.836736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:33.836695    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"
Jul 17 11:44:33.874422 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:33.874392    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ed96aa-77fd-4b21-aac3-77429e5ebd0d-maas-api-service-ca\") pod \"33ed96aa-77fd-4b21-aac3-77429e5ebd0d\" (UID: \"33ed96aa-77fd-4b21-aac3-77429e5ebd0d\") "
Jul 17 11:44:33.874736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:33.874693    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33ed96aa-77fd-4b21-aac3-77429e5ebd0d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33ed96aa-77fd-4b21-aac3-77429e5ebd0d" (UID: "33ed96aa-77fd-4b21-aac3-77429e5ebd0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:44:33.975978 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:33.975890    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ed96aa-77fd-4b21-aac3-77429e5ebd0d-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:44:34.680079 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:34.680049    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"
Jul 17 11:44:34.680293 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:34.680093    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4" event={"ID":"33ed96aa-77fd-4b21-aac3-77429e5ebd0d","Type":"ContainerDied","Data":"de5fe488f9b9acdf004dfc9ad4582390c1605515ff2e995ebb17bf4cb0db331f"}
Jul 17 11:44:34.680293 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:34.680162    2566 scope.go:117] "RemoveContainer" containerID="5b20a9cce4e5cc6816002a1a2325f7ad3d557c9a2c33379f617997881ab3604d"
Jul 17 11:44:34.705837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:34.705808    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"]
Jul 17 11:44:34.719677 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:34.719644    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-6p7t4"]
Jul 17 11:44:35.657152 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:35.657119    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" path="/var/lib/kubelet/pods/33ed96aa-77fd-4b21-aac3-77429e5ebd0d/volumes"
Jul 17 11:44:44.046485 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:44.046449    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:44:45.652118 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:45.652087    2566 scope.go:117] "RemoveContainer" containerID="9b7f41bed7b917bed12213c00ce09e2b28183ea21631dff5722f25c02d30bda8"
Jul 17 11:44:46.739910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:46.739823    2566 generic.go:356] "Generic (PLEG): container finished" podID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerID="7e97446a4851cc8a8130d2d9887cabb933904d369ee7a9ec31211e172aaa31a4" exitCode=6
Jul 17 11:44:46.739910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:46.739896    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" event={"ID":"aa75b23a-ed6b-4a75-a123-8c8000524d1a","Type":"ContainerDied","Data":"7e97446a4851cc8a8130d2d9887cabb933904d369ee7a9ec31211e172aaa31a4"}
Jul 17 11:44:46.740435 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:46.739956    2566 scope.go:117] "RemoveContainer" containerID="9b7f41bed7b917bed12213c00ce09e2b28183ea21631dff5722f25c02d30bda8"
Jul 17 11:44:46.740435 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:46.740271    2566 scope.go:117] "RemoveContainer" containerID="7e97446a4851cc8a8130d2d9887cabb933904d369ee7a9ec31211e172aaa31a4"
Jul 17 11:44:46.740552 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:46.740496    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-zrstp_odh-ai-gateway-infra(aa75b23a-ed6b-4a75-a123-8c8000524d1a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a"
Jul 17 11:44:47.653887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:47.653854    2566 scope.go:117] "RemoveContainer" containerID="3c3bc646d5fdf69ec1e80c5ed73e1659c2997511b9281c8fbc7a3c5f9b41a9b0"
Jul 17 11:44:47.779772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:47.779741    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"]
Jul 17 11:44:47.912307 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:47.912250    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"
Jul 17 11:44:48.008339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.008312    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa75b23a-ed6b-4a75-a123-8c8000524d1a-maas-api-service-ca\") pod \"aa75b23a-ed6b-4a75-a123-8c8000524d1a\" (UID: \"aa75b23a-ed6b-4a75-a123-8c8000524d1a\") "
Jul 17 11:44:48.008678 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.008655    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa75b23a-ed6b-4a75-a123-8c8000524d1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aa75b23a-ed6b-4a75-a123-8c8000524d1a" (UID: "aa75b23a-ed6b-4a75-a123-8c8000524d1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:44:48.109694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.109670    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa75b23a-ed6b-4a75-a123-8c8000524d1a-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:44:48.752349 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.752319    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"
Jul 17 11:44:48.752349 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.752338    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp" event={"ID":"aa75b23a-ed6b-4a75-a123-8c8000524d1a","Type":"ContainerDied","Data":"8ab10bf242ea45f248515e40466af1aebe982f3cbf2d658df8cb57b8ed2b9b9d"}
Jul 17 11:44:48.752614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.752401    2566 scope.go:117] "RemoveContainer" containerID="7e97446a4851cc8a8130d2d9887cabb933904d369ee7a9ec31211e172aaa31a4"
Jul 17 11:44:48.754540 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.754518    2566 generic.go:356] "Generic (PLEG): container finished" podID="04731422-78fc-4fbd-a19b-40140651f4e1" containerID="d835e687355b36fd30f6e1e619813bc266a4909a3c7577c7e4856ece8f1f029e" exitCode=6
Jul 17 11:44:48.754679 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.754549    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw" event={"ID":"04731422-78fc-4fbd-a19b-40140651f4e1","Type":"ContainerDied","Data":"d835e687355b36fd30f6e1e619813bc266a4909a3c7577c7e4856ece8f1f029e"}
Jul 17 11:44:48.754830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.754815    2566 scope.go:117] "RemoveContainer" containerID="d835e687355b36fd30f6e1e619813bc266a4909a3c7577c7e4856ece8f1f029e"
Jul 17 11:44:48.755023 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:44:48.755005    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw_odh-ai-gateway-infra(04731422-78fc-4fbd-a19b-40140651f4e1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw" podUID="04731422-78fc-4fbd-a19b-40140651f4e1"
Jul 17 11:44:48.761951 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.761928    2566 scope.go:117] "RemoveContainer" containerID="3c3bc646d5fdf69ec1e80c5ed73e1659c2997511b9281c8fbc7a3c5f9b41a9b0"
Jul 17 11:44:48.792769 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.792739    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"]
Jul 17 11:44:48.799878 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:48.799851    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-zrstp"]
Jul 17 11:44:49.657823 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:49.657783    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" path="/var/lib/kubelet/pods/aa75b23a-ed6b-4a75-a123-8c8000524d1a/volumes"
Jul 17 11:44:49.787773 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:49.787745    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"]
Jul 17 11:44:49.926556 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:49.926530    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"
Jul 17 11:44:50.025398 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:50.025369    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04731422-78fc-4fbd-a19b-40140651f4e1-maas-api-service-ca\") pod \"04731422-78fc-4fbd-a19b-40140651f4e1\" (UID: \"04731422-78fc-4fbd-a19b-40140651f4e1\") "
Jul 17 11:44:50.025772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:50.025749    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04731422-78fc-4fbd-a19b-40140651f4e1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "04731422-78fc-4fbd-a19b-40140651f4e1" (UID: "04731422-78fc-4fbd-a19b-40140651f4e1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:44:50.127023 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:50.126951    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04731422-78fc-4fbd-a19b-40140651f4e1-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:44:50.767419 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:50.767383    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw" event={"ID":"04731422-78fc-4fbd-a19b-40140651f4e1","Type":"ContainerDied","Data":"491839590d1e4203fbbdbefb7a8f72367d66e79ecf64d355705589788852a19b"}
Jul 17 11:44:50.767613 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:50.767445    2566 scope.go:117] "RemoveContainer" containerID="d835e687355b36fd30f6e1e619813bc266a4909a3c7577c7e4856ece8f1f029e"
Jul 17 11:44:50.767613 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:50.767401    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"
Jul 17 11:44:50.791829 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:50.791806    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"]
Jul 17 11:44:50.794661 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:50.794642    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-5q4lw"]
Jul 17 11:44:51.657397 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:51.657369    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" path="/var/lib/kubelet/pods/04731422-78fc-4fbd-a19b-40140651f4e1/volumes"
Jul 17 11:44:58.873500 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873469    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"]
Jul 17 11:44:58.873874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873828    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerName="revoke-keys"
Jul 17 11:44:58.873874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873839    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerName="revoke-keys"
Jul 17 11:44:58.873874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873849    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" containerName="revoke-keys"
Jul 17 11:44:58.873874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873854    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" containerName="revoke-keys"
Jul 17 11:44:58.873874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873865    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerName="revoke-keys"
Jul 17 11:44:58.873874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873873    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873884    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873889    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873896    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873901    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873911    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873917    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873966    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873973    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873979    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873985    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873991    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.873997    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerName="revoke-keys"
Jul 17 11:44:58.874070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.874001    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" containerName="revoke-keys"
Jul 17 11:44:58.878489 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.878474    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"
Jul 17 11:44:58.881505 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.881482    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:44:58.881607 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.881511    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:44:58.889525 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.889501    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"]
Jul 17 11:44:58.902715 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:58.902674    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7baa2993-caa5-4227-9362-14a8dd021ac9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd\" (UID: \"7baa2993-caa5-4227-9362-14a8dd021ac9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"
Jul 17 11:44:59.003941 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:59.003911    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7baa2993-caa5-4227-9362-14a8dd021ac9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd\" (UID: \"7baa2993-caa5-4227-9362-14a8dd021ac9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"
Jul 17 11:44:59.004498 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:59.004479    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7baa2993-caa5-4227-9362-14a8dd021ac9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd\" (UID: \"7baa2993-caa5-4227-9362-14a8dd021ac9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"
Jul 17 11:44:59.188923 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:59.188853    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"
Jul 17 11:44:59.324086 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:59.324026    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"]
Jul 17 11:44:59.329213 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:44:59.329178    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7baa2993_caa5_4227_9362_14a8dd021ac9.slice/crio-cc0463ace4a6600d905648917051f3d8752090e807d61f7e7c6d6aaea0384110 WatchSource:0}: Error finding container cc0463ace4a6600d905648917051f3d8752090e807d61f7e7c6d6aaea0384110: Status 404 returned error can't find the container with id cc0463ace4a6600d905648917051f3d8752090e807d61f7e7c6d6aaea0384110
Jul 17 11:44:59.818573 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:59.818471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" event={"ID":"7baa2993-caa5-4227-9362-14a8dd021ac9","Type":"ContainerStarted","Data":"9fa24d2b932d65b4c0331f7cec62f324fab122a59ac5c96c2a1875772e773a9f"}
Jul 17 11:44:59.818573 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:59.818529    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" event={"ID":"7baa2993-caa5-4227-9362-14a8dd021ac9","Type":"ContainerStarted","Data":"cc0463ace4a6600d905648917051f3d8752090e807d61f7e7c6d6aaea0384110"}
Jul 17 11:44:59.839908 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:44:59.839859    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" podStartSLOduration=1.839845057 podStartE2EDuration="1.839845057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:44:59.837632748 +0000 UTC m=+2882.694716692" watchObservedRunningTime="2026-07-17 11:44:59.839845057 +0000 UTC m=+2882.696929001"
Jul 17 11:45:00.145443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.145410    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj"]
Jul 17 11:45:00.145830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.145804    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerName="revoke-keys"
Jul 17 11:45:00.145830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.145816    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ed96aa-77fd-4b21-aac3-77429e5ebd0d" containerName="revoke-keys"
Jul 17 11:45:00.145830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.145824    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" containerName="revoke-keys"
Jul 17 11:45:00.145830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.145829    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" containerName="revoke-keys"
Jul 17 11:45:00.145967 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.145841    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerName="revoke-keys"
Jul 17 11:45:00.145967 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.145847    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerName="revoke-keys"
Jul 17 11:45:00.145967 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.145907    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa75b23a-ed6b-4a75-a123-8c8000524d1a" containerName="revoke-keys"
Jul 17 11:45:00.145967 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.145915    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="04731422-78fc-4fbd-a19b-40140651f4e1" containerName="revoke-keys"
Jul 17 11:45:00.148898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.148878    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l"]
Jul 17 11:45:00.149008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.148991    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj"
Jul 17 11:45:00.151995 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.151979    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz"]
Jul 17 11:45:00.152101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.152086    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l"
Jul 17 11:45:00.155106 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.155086    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz"
Jul 17 11:45:00.157457 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.157433    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj"]
Jul 17 11:45:00.162108 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.162051    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l"]
Jul 17 11:45:00.162582 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.162561    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj"
Jul 17 11:45:00.164572 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.164551    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz"]
Jul 17 11:45:00.172016 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.172000    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l"
Jul 17 11:45:00.179658 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.179631    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz"
Jul 17 11:45:00.515299 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.515231    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"]
Jul 17 11:45:00.519917 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.519898    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"
Jul 17 11:45:00.528534 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.528461    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"]
Jul 17 11:45:00.535374 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.535294    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj"]
Jul 17 11:45:00.538296 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:00.538268    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20cb8118_63a1_4a98_a625_91442c941d88.slice/crio-7a77678d7e538c357dc9b271c54115c37dc74ae0098213f9541faf7b860a9446 WatchSource:0}: Error finding container 7a77678d7e538c357dc9b271c54115c37dc74ae0098213f9541faf7b860a9446: Status 404 returned error can't find the container with id 7a77678d7e538c357dc9b271c54115c37dc74ae0098213f9541faf7b860a9446
Jul 17 11:45:00.560612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.560588    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l"]
Jul 17 11:45:00.564464 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:00.564437    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b64a8ee_ed55_4b9e_adf4_435230659a99.slice/crio-f0645dd3d9024f16b0e92fc2d9d162e471007850e98ace8beb8eeec83f43c573 WatchSource:0}: Error finding container f0645dd3d9024f16b0e92fc2d9d162e471007850e98ace8beb8eeec83f43c573: Status 404 returned error can't find the container with id f0645dd3d9024f16b0e92fc2d9d162e471007850e98ace8beb8eeec83f43c573
Jul 17 11:45:00.564948 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.564930    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz"]
Jul 17 11:45:00.572770 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:00.572744    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae23873_c392_4119_9128_e4e633040407.slice/crio-0c452de5d95fa73ae825089e76209d38245f843b20777448b3e7bf36b23213aa WatchSource:0}: Error finding container 0c452de5d95fa73ae825089e76209d38245f843b20777448b3e7bf36b23213aa: Status 404 returned error can't find the container with id 0c452de5d95fa73ae825089e76209d38245f843b20777448b3e7bf36b23213aa
Jul 17 11:45:00.620546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.620526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv\" (UID: \"cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"
Jul 17 11:45:00.721916 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.721874    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv\" (UID: \"cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"
Jul 17 11:45:00.722495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.722474    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv\" (UID: \"cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"
Jul 17 11:45:00.824475 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.824447    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l" event={"ID":"7b64a8ee-ed55-4b9e-adf4-435230659a99","Type":"ContainerStarted","Data":"f0645dd3d9024f16b0e92fc2d9d162e471007850e98ace8beb8eeec83f43c573"}
Jul 17 11:45:00.825574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.825545    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz" event={"ID":"4ae23873-c392-4119-9128-e4e633040407","Type":"ContainerStarted","Data":"0c452de5d95fa73ae825089e76209d38245f843b20777448b3e7bf36b23213aa"}
Jul 17 11:45:00.826590 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.826557    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj" event={"ID":"20cb8118-63a1-4a98-a625-91442c941d88","Type":"ContainerStarted","Data":"7a77678d7e538c357dc9b271c54115c37dc74ae0098213f9541faf7b860a9446"}
Jul 17 11:45:00.828338 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.828310    2566 generic.go:356] "Generic (PLEG): container finished" podID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerID="9fa24d2b932d65b4c0331f7cec62f324fab122a59ac5c96c2a1875772e773a9f" exitCode=6
Jul 17 11:45:00.828427 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.828374    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" event={"ID":"7baa2993-caa5-4227-9362-14a8dd021ac9","Type":"ContainerDied","Data":"9fa24d2b932d65b4c0331f7cec62f324fab122a59ac5c96c2a1875772e773a9f"}
Jul 17 11:45:00.828595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.828576    2566 scope.go:117] "RemoveContainer" containerID="9fa24d2b932d65b4c0331f7cec62f324fab122a59ac5c96c2a1875772e773a9f"
Jul 17 11:45:00.833093 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.833078    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"
Jul 17 11:45:00.976818 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:00.976795    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"]
Jul 17 11:45:00.978941 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:00.978915    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbaa2e44_d2ed_4c68_b0e0_b5ceaa941986.slice/crio-9d9096a8d44c066a827171f57bcf531576667acb3a0e02663a1812b0903e5429 WatchSource:0}: Error finding container 9d9096a8d44c066a827171f57bcf531576667acb3a0e02663a1812b0903e5429: Status 404 returned error can't find the container with id 9d9096a8d44c066a827171f57bcf531576667acb3a0e02663a1812b0903e5429
Jul 17 11:45:01.835594 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.835565    2566 generic.go:356] "Generic (PLEG): container finished" podID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerID="8decd4a1fd9a222acbfde109bc12f5a23f9431c13bbba1749e98a1091f7f830a" exitCode=6
Jul 17 11:45:01.835986 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.835656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" event={"ID":"7baa2993-caa5-4227-9362-14a8dd021ac9","Type":"ContainerDied","Data":"8decd4a1fd9a222acbfde109bc12f5a23f9431c13bbba1749e98a1091f7f830a"}
Jul 17 11:45:01.835986 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.835750    2566 scope.go:117] "RemoveContainer" containerID="9fa24d2b932d65b4c0331f7cec62f324fab122a59ac5c96c2a1875772e773a9f"
Jul 17 11:45:01.835986 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.835862    2566 scope.go:117] "RemoveContainer" containerID="8decd4a1fd9a222acbfde109bc12f5a23f9431c13bbba1749e98a1091f7f830a"
Jul 17 11:45:01.836111 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:01.836051    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd_odh-ai-gateway-infra(7baa2993-caa5-4227-9362-14a8dd021ac9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9"
Jul 17 11:45:01.837169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.837148    2566 generic.go:356] "Generic (PLEG): container finished" podID="cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986" containerID="26d00931efabd5c36af9c800b0b7fe9ddfeb899451d29d8fb4b745028423b9ae" exitCode=0
Jul 17 11:45:01.837272 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.837227    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv" event={"ID":"cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986","Type":"ContainerDied","Data":"26d00931efabd5c36af9c800b0b7fe9ddfeb899451d29d8fb4b745028423b9ae"}
Jul 17 11:45:01.837330 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.837277    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv" event={"ID":"cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986","Type":"ContainerStarted","Data":"9d9096a8d44c066a827171f57bcf531576667acb3a0e02663a1812b0903e5429"}
Jul 17 11:45:01.838759 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.838738    2566 generic.go:356] "Generic (PLEG): container finished" podID="7b64a8ee-ed55-4b9e-adf4-435230659a99" containerID="a5967290215d8da0602717d37709ba1d76238de44fcadd43ce55a0f89db6b8ea" exitCode=0
Jul 17 11:45:01.838853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.838764    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l" event={"ID":"7b64a8ee-ed55-4b9e-adf4-435230659a99","Type":"ContainerDied","Data":"a5967290215d8da0602717d37709ba1d76238de44fcadd43ce55a0f89db6b8ea"}
Jul 17 11:45:01.840292 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.840275    2566 generic.go:356] "Generic (PLEG): container finished" podID="4ae23873-c392-4119-9128-e4e633040407" containerID="1fe01f2a3c096c71d2ccaf65fc17bed3166e1f1830d5b5728b3a3194da89778f" exitCode=0
Jul 17 11:45:01.840394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.840299    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz" event={"ID":"4ae23873-c392-4119-9128-e4e633040407","Type":"ContainerDied","Data":"1fe01f2a3c096c71d2ccaf65fc17bed3166e1f1830d5b5728b3a3194da89778f"}
Jul 17 11:45:01.841899 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.841880    2566 generic.go:356] "Generic (PLEG): container finished" podID="20cb8118-63a1-4a98-a625-91442c941d88" containerID="ed4fa8e1c6e70d26acd6416bb25a8f13e560db58207a1b22d21fddbb3103f2db" exitCode=0
Jul 17 11:45:01.841987 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:01.841905    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj" event={"ID":"20cb8118-63a1-4a98-a625-91442c941d88","Type":"ContainerDied","Data":"ed4fa8e1c6e70d26acd6416bb25a8f13e560db58207a1b22d21fddbb3103f2db"}
Jul 17 11:45:02.849955 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:02.849919    2566 scope.go:117] "RemoveContainer" containerID="8decd4a1fd9a222acbfde109bc12f5a23f9431c13bbba1749e98a1091f7f830a"
Jul 17 11:45:02.850491 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:02.850108    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd_odh-ai-gateway-infra(7baa2993-caa5-4227-9362-14a8dd021ac9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9"
Jul 17 11:45:02.994416 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:02.994393    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"
Jul 17 11:45:03.042741 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.042510    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986-maas-api-service-ca\") pod \"cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986\" (UID: \"cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986\") "
Jul 17 11:45:03.045491 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.043527    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986" (UID: "cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:45:03.062212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.062191    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz"
Jul 17 11:45:03.080668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.080644    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj"
Jul 17 11:45:03.087959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.087938    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l"
Jul 17 11:45:03.144742 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.144659    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:03.856178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.856151    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz"
Jul 17 11:45:03.856582 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.856150    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz" event={"ID":"4ae23873-c392-4119-9128-e4e633040407","Type":"ContainerDied","Data":"0c452de5d95fa73ae825089e76209d38245f843b20777448b3e7bf36b23213aa"}
Jul 17 11:45:03.856582 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.856287    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c452de5d95fa73ae825089e76209d38245f843b20777448b3e7bf36b23213aa"
Jul 17 11:45:03.857503 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.857484    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj"
Jul 17 11:45:03.857654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.857534    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-42ntj" event={"ID":"20cb8118-63a1-4a98-a625-91442c941d88","Type":"ContainerDied","Data":"7a77678d7e538c357dc9b271c54115c37dc74ae0098213f9541faf7b860a9446"}
Jul 17 11:45:03.857654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.857583    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a77678d7e538c357dc9b271c54115c37dc74ae0098213f9541faf7b860a9446"
Jul 17 11:45:03.858941 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.858925    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv"
Jul 17 11:45:03.859061 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.858948    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv" event={"ID":"cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986","Type":"ContainerDied","Data":"9d9096a8d44c066a827171f57bcf531576667acb3a0e02663a1812b0903e5429"}
Jul 17 11:45:03.859061 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.858995    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d9096a8d44c066a827171f57bcf531576667acb3a0e02663a1812b0903e5429"
Jul 17 11:45:03.860440 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.860417    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l" event={"ID":"7b64a8ee-ed55-4b9e-adf4-435230659a99","Type":"ContainerDied","Data":"f0645dd3d9024f16b0e92fc2d9d162e471007850e98ace8beb8eeec83f43c573"}
Jul 17 11:45:03.860440 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.860437    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l"
Jul 17 11:45:03.860610 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:03.860458    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0645dd3d9024f16b0e92fc2d9d162e471007850e98ace8beb8eeec83f43c573"
Jul 17 11:45:05.106837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.106807    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:45:05.626600 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.626571    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"]
Jul 17 11:45:05.627031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627014    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20cb8118-63a1-4a98-a625-91442c941d88" containerName="cleanup"
Jul 17 11:45:05.627136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627033    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="20cb8118-63a1-4a98-a625-91442c941d88" containerName="cleanup"
Jul 17 11:45:05.627136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627060    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986" containerName="revoke-keys"
Jul 17 11:45:05.627136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627068    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986" containerName="revoke-keys"
Jul 17 11:45:05.627136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627100    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae23873-c392-4119-9128-e4e633040407" containerName="cleanup"
Jul 17 11:45:05.627136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627109    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae23873-c392-4119-9128-e4e633040407" containerName="cleanup"
Jul 17 11:45:05.627136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627121    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b64a8ee-ed55-4b9e-adf4-435230659a99" containerName="cleanup"
Jul 17 11:45:05.627136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627129    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b64a8ee-ed55-4b9e-adf4-435230659a99" containerName="cleanup"
Jul 17 11:45:05.627472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627222    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="20cb8118-63a1-4a98-a625-91442c941d88" containerName="cleanup"
Jul 17 11:45:05.627472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627235    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ae23873-c392-4119-9128-e4e633040407" containerName="cleanup"
Jul 17 11:45:05.627472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627244    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986" containerName="revoke-keys"
Jul 17 11:45:05.627472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.627252    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b64a8ee-ed55-4b9e-adf4-435230659a99" containerName="cleanup"
Jul 17 11:45:05.630426 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.630408    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"
Jul 17 11:45:05.640512 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.640491    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"]
Jul 17 11:45:05.665034 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.665004    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-24qw2\" (UID: \"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"
Jul 17 11:45:05.766436 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.766396    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-24qw2\" (UID: \"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"
Jul 17 11:45:05.767033 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.767015    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-24qw2\" (UID: \"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"
Jul 17 11:45:05.942115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:05.942036    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"
Jul 17 11:45:06.073003 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:06.072975    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"]
Jul 17 11:45:06.074634 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:06.074601    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2f2d5d9_7ec1_456d_ac66_b08ec7bc7459.slice/crio-b3781b61323b56aa8eb73e35de3d776a0e041197358771d6ce512bd863978baf WatchSource:0}: Error finding container b3781b61323b56aa8eb73e35de3d776a0e041197358771d6ce512bd863978baf: Status 404 returned error can't find the container with id b3781b61323b56aa8eb73e35de3d776a0e041197358771d6ce512bd863978baf
Jul 17 11:45:06.884409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:06.884325    2566 generic.go:356] "Generic (PLEG): container finished" podID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerID="a51b369054ca109186c21b6441f8ada839c3461fabcdd3ba2ea2b53dc5e54372" exitCode=6
Jul 17 11:45:06.884409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:06.884378    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" event={"ID":"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459","Type":"ContainerDied","Data":"a51b369054ca109186c21b6441f8ada839c3461fabcdd3ba2ea2b53dc5e54372"}
Jul 17 11:45:06.884852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:06.884436    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" event={"ID":"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459","Type":"ContainerStarted","Data":"b3781b61323b56aa8eb73e35de3d776a0e041197358771d6ce512bd863978baf"}
Jul 17 11:45:06.884852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:06.884599    2566 scope.go:117] "RemoveContainer" containerID="a51b369054ca109186c21b6441f8ada839c3461fabcdd3ba2ea2b53dc5e54372"
Jul 17 11:45:07.891897 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:07.891864    2566 generic.go:356] "Generic (PLEG): container finished" podID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerID="01b9b3ca4a616fa58da17a63dc4dbe2694adabb1b014a53a36ee2b89b3508fd3" exitCode=6
Jul 17 11:45:07.892365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:07.891912    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" event={"ID":"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459","Type":"ContainerDied","Data":"01b9b3ca4a616fa58da17a63dc4dbe2694adabb1b014a53a36ee2b89b3508fd3"}
Jul 17 11:45:07.892365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:07.891955    2566 scope.go:117] "RemoveContainer" containerID="a51b369054ca109186c21b6441f8ada839c3461fabcdd3ba2ea2b53dc5e54372"
Jul 17 11:45:07.892365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:07.892090    2566 scope.go:117] "RemoveContainer" containerID="01b9b3ca4a616fa58da17a63dc4dbe2694adabb1b014a53a36ee2b89b3508fd3"
Jul 17 11:45:07.892365 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:07.892328    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-24qw2_odh-ai-gateway-infra(f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459"
Jul 17 11:45:08.898399 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:08.898372    2566 scope.go:117] "RemoveContainer" containerID="01b9b3ca4a616fa58da17a63dc4dbe2694adabb1b014a53a36ee2b89b3508fd3"
Jul 17 11:45:08.898847 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:08.898568    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-24qw2_odh-ai-gateway-infra(f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459"
Jul 17 11:45:11.145854 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:11.145820    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l"]
Jul 17 11:45:11.153205 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:11.153181    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fc3287-29738145-g5n5l"]
Jul 17 11:45:11.658246 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:11.658211    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b64a8ee-ed55-4b9e-adf4-435230659a99" path="/var/lib/kubelet/pods/7b64a8ee-ed55-4b9e-adf4-435230659a99/volumes"
Jul 17 11:45:15.652198 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:15.652165    2566 scope.go:117] "RemoveContainer" containerID="8decd4a1fd9a222acbfde109bc12f5a23f9431c13bbba1749e98a1091f7f830a"
Jul 17 11:45:15.936408 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:15.936329    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" event={"ID":"7baa2993-caa5-4227-9362-14a8dd021ac9","Type":"ContainerStarted","Data":"ae814e936d817519a6ccd7fc30bf552041502d342ef600e9fe92edc722efe2b0"}
Jul 17 11:45:16.942865 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:16.942834    2566 generic.go:356] "Generic (PLEG): container finished" podID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerID="ae814e936d817519a6ccd7fc30bf552041502d342ef600e9fe92edc722efe2b0" exitCode=6
Jul 17 11:45:16.943266 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:16.942907    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" event={"ID":"7baa2993-caa5-4227-9362-14a8dd021ac9","Type":"ContainerDied","Data":"ae814e936d817519a6ccd7fc30bf552041502d342ef600e9fe92edc722efe2b0"}
Jul 17 11:45:16.943266 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:16.942969    2566 scope.go:117] "RemoveContainer" containerID="8decd4a1fd9a222acbfde109bc12f5a23f9431c13bbba1749e98a1091f7f830a"
Jul 17 11:45:16.943266 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:16.943216    2566 scope.go:117] "RemoveContainer" containerID="ae814e936d817519a6ccd7fc30bf552041502d342ef600e9fe92edc722efe2b0"
Jul 17 11:45:16.943460 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:16.943436    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd_odh-ai-gateway-infra(7baa2993-caa5-4227-9362-14a8dd021ac9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9"
Jul 17 11:45:16.969466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:16.969444    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"]
Jul 17 11:45:18.088473 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:18.088443    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"
Jul 17 11:45:18.175953 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:18.175921    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7baa2993-caa5-4227-9362-14a8dd021ac9-maas-api-service-ca\") pod \"7baa2993-caa5-4227-9362-14a8dd021ac9\" (UID: \"7baa2993-caa5-4227-9362-14a8dd021ac9\") "
Jul 17 11:45:18.176307 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:18.176283    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7baa2993-caa5-4227-9362-14a8dd021ac9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7baa2993-caa5-4227-9362-14a8dd021ac9" (UID: "7baa2993-caa5-4227-9362-14a8dd021ac9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:45:18.277071 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:18.276996    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7baa2993-caa5-4227-9362-14a8dd021ac9-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:18.955828 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:18.955797    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"
Jul 17 11:45:18.956036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:18.955835    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd" event={"ID":"7baa2993-caa5-4227-9362-14a8dd021ac9","Type":"ContainerDied","Data":"cc0463ace4a6600d905648917051f3d8752090e807d61f7e7c6d6aaea0384110"}
Jul 17 11:45:18.956036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:18.955899    2566 scope.go:117] "RemoveContainer" containerID="ae814e936d817519a6ccd7fc30bf552041502d342ef600e9fe92edc722efe2b0"
Jul 17 11:45:18.984858 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:18.983044    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"]
Jul 17 11:45:18.988497 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:18.988470    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-vwdwd"]
Jul 17 11:45:19.657166 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:19.657132    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" path="/var/lib/kubelet/pods/7baa2993-caa5-4227-9362-14a8dd021ac9/volumes"
Jul 17 11:45:20.652198 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:20.652174    2566 scope.go:117] "RemoveContainer" containerID="01b9b3ca4a616fa58da17a63dc4dbe2694adabb1b014a53a36ee2b89b3508fd3"
Jul 17 11:45:20.971162 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:20.971075    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" event={"ID":"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459","Type":"ContainerStarted","Data":"0084d79436f134ddebb1c36e5cb44ad4cb25466fb3e9e9d9885e77af5b071b65"}
Jul 17 11:45:20.991565 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:20.991516    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" podStartSLOduration=15.991503413 podStartE2EDuration="15.991503413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:45:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:45:20.989668449 +0000 UTC m=+2903.846752394" watchObservedRunningTime="2026-07-17 11:45:20.991503413 +0000 UTC m=+2903.848587357"
Jul 17 11:45:21.978614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:21.978583    2566 generic.go:356] "Generic (PLEG): container finished" podID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerID="0084d79436f134ddebb1c36e5cb44ad4cb25466fb3e9e9d9885e77af5b071b65" exitCode=6
Jul 17 11:45:21.979053 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:21.978655    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" event={"ID":"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459","Type":"ContainerDied","Data":"0084d79436f134ddebb1c36e5cb44ad4cb25466fb3e9e9d9885e77af5b071b65"}
Jul 17 11:45:21.979053 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:21.978737    2566 scope.go:117] "RemoveContainer" containerID="01b9b3ca4a616fa58da17a63dc4dbe2694adabb1b014a53a36ee2b89b3508fd3"
Jul 17 11:45:21.979158 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:21.979056    2566 scope.go:117] "RemoveContainer" containerID="0084d79436f134ddebb1c36e5cb44ad4cb25466fb3e9e9d9885e77af5b071b65"
Jul 17 11:45:21.979296 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:21.979278    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-24qw2_odh-ai-gateway-infra(f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459"
Jul 17 11:45:22.004370 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:22.004342    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"]
Jul 17 11:45:22.366633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:22.366601    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:45:23.118590 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:23.118569    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"
Jul 17 11:45:23.223661 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:23.223627    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459-maas-api-service-ca\") pod \"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459\" (UID: \"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459\") "
Jul 17 11:45:23.224044 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:23.224025    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" (UID: "f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:45:23.325041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:23.325008    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:23.992160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:23.992075    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"
Jul 17 11:45:23.992160 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:23.992131    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2" event={"ID":"f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459","Type":"ContainerDied","Data":"b3781b61323b56aa8eb73e35de3d776a0e041197358771d6ce512bd863978baf"}
Jul 17 11:45:23.992361 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:23.992185    2566 scope.go:117] "RemoveContainer" containerID="0084d79436f134ddebb1c36e5cb44ad4cb25466fb3e9e9d9885e77af5b071b65"
Jul 17 11:45:24.025489 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:24.025460    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"]
Jul 17 11:45:24.039612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:24.039577    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-24qw2"]
Jul 17 11:45:25.657088 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:25.657049    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" path="/var/lib/kubelet/pods/f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459/volumes"
Jul 17 11:45:26.411689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.411660    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"]
Jul 17 11:45:26.411878 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.411855    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy" containerID="cri-o://e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e" gracePeriod=30
Jul 17 11:45:26.777316 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777219    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"]
Jul 17 11:45:26.777808 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777790    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerName="revoke-keys"
Jul 17 11:45:26.777894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777811    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerName="revoke-keys"
Jul 17 11:45:26.777894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777829    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerName="revoke-keys"
Jul 17 11:45:26.777894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777837    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerName="revoke-keys"
Jul 17 11:45:26.777894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777846    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerName="revoke-keys"
Jul 17 11:45:26.777894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777854    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerName="revoke-keys"
Jul 17 11:45:26.777894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777870    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerName="revoke-keys"
Jul 17 11:45:26.777894 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777880    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerName="revoke-keys"
Jul 17 11:45:26.778233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.777992    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerName="revoke-keys"
Jul 17 11:45:26.778233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.778005    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerName="revoke-keys"
Jul 17 11:45:26.778233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.778016    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerName="revoke-keys"
Jul 17 11:45:26.778233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.778046    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerName="revoke-keys"
Jul 17 11:45:26.778233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.778057    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerName="revoke-keys"
Jul 17 11:45:26.778233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.778068    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerName="revoke-keys"
Jul 17 11:45:26.782547 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.782526    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"
Jul 17 11:45:26.785559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.785531    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:45:26.785680 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.785604    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:45:26.789309 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.789286    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"]
Jul 17 11:45:26.850271 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.850245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c62ac0-771c-49c4-b542-6aad30b89b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8\" (UID: \"00c62ac0-771c-49c4-b542-6aad30b89b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"
Jul 17 11:45:26.951288 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.951246    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c62ac0-771c-49c4-b542-6aad30b89b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8\" (UID: \"00c62ac0-771c-49c4-b542-6aad30b89b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"
Jul 17 11:45:26.951884 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:26.951865    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c62ac0-771c-49c4-b542-6aad30b89b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8\" (UID: \"00c62ac0-771c-49c4-b542-6aad30b89b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"
Jul 17 11:45:27.094650 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:27.094622    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"
Jul 17 11:45:27.236556 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:27.236529    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"]
Jul 17 11:45:27.238747 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:27.238698    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c62ac0_771c_49c4_b542_6aad30b89b57.slice/crio-6dafe0c0ef522cf0f013813064b03fc18f804afbe689fcb457bd3369c2705066 WatchSource:0}: Error finding container 6dafe0c0ef522cf0f013813064b03fc18f804afbe689fcb457bd3369c2705066: Status 404 returned error can't find the container with id 6dafe0c0ef522cf0f013813064b03fc18f804afbe689fcb457bd3369c2705066
Jul 17 11:45:27.909735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:27.909696    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"]
Jul 17 11:45:27.910085 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:27.910075    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerName="revoke-keys"
Jul 17 11:45:27.910141 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:27.910086    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f2d5d9-7ec1-456d-ac66-b08ec7bc7459" containerName="revoke-keys"
Jul 17 11:45:27.910141 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:27.910099    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerName="revoke-keys"
Jul 17 11:45:27.910141 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:27.910104    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7baa2993-caa5-4227-9362-14a8dd021ac9" containerName="revoke-keys"
Jul 17 11:45:27.913023 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:27.913007    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"
Jul 17 11:45:27.921268 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:27.921237    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"]
Jul 17 11:45:28.013010 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.012969    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8" event={"ID":"00c62ac0-771c-49c4-b542-6aad30b89b57","Type":"ContainerStarted","Data":"bd3a76c10064e20f80d33f0cbae5e923e4349beea127776c237ae9ccc90ad8f9"}
Jul 17 11:45:28.013172 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.013022    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8" event={"ID":"00c62ac0-771c-49c4-b542-6aad30b89b57","Type":"ContainerStarted","Data":"6dafe0c0ef522cf0f013813064b03fc18f804afbe689fcb457bd3369c2705066"}
Jul 17 11:45:28.034019 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.033967    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8" podStartSLOduration=2.033949496 podStartE2EDuration="2.033949496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:45:28.033548632 +0000 UTC m=+2910.890632604" watchObservedRunningTime="2026-07-17 11:45:28.033949496 +0000 UTC m=+2910.891033445"
Jul 17 11:45:28.061187 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.061163    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3ce2e0-ccaa-4975-a7f0-6e1872339e13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-kzfrc\" (UID: \"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"
Jul 17 11:45:28.162559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.162481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3ce2e0-ccaa-4975-a7f0-6e1872339e13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-kzfrc\" (UID: \"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"
Jul 17 11:45:28.163069 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.163048    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3ce2e0-ccaa-4975-a7f0-6e1872339e13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-kzfrc\" (UID: \"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"
Jul 17 11:45:28.224057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.224026    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"
Jul 17 11:45:28.356252 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.356225    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"]
Jul 17 11:45:28.358018 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:28.357991    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3ce2e0_ccaa_4975_a7f0_6e1872339e13.slice/crio-4183e01437950f41c09bd7989c708b58b5210bff15af72047befabe26e371d90 WatchSource:0}: Error finding container 4183e01437950f41c09bd7989c708b58b5210bff15af72047befabe26e371d90: Status 404 returned error can't find the container with id 4183e01437950f41c09bd7989c708b58b5210bff15af72047befabe26e371d90
Jul 17 11:45:28.890196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.890158    2566 patch_prober.go:28] interesting pod/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 11:45:28.890356 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.890218    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 11:45:28.990220 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.990190    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"]
Jul 17 11:45:28.993844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:28.993823    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"
Jul 17 11:45:29.004324 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.004295    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"]
Jul 17 11:45:29.020832 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.020803    2566 generic.go:356] "Generic (PLEG): container finished" podID="00c62ac0-771c-49c4-b542-6aad30b89b57" containerID="bd3a76c10064e20f80d33f0cbae5e923e4349beea127776c237ae9ccc90ad8f9" exitCode=0
Jul 17 11:45:29.020986 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.020887    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8" event={"ID":"00c62ac0-771c-49c4-b542-6aad30b89b57","Type":"ContainerDied","Data":"bd3a76c10064e20f80d33f0cbae5e923e4349beea127776c237ae9ccc90ad8f9"}
Jul 17 11:45:29.022434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.022413    2566 generic.go:356] "Generic (PLEG): container finished" podID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerID="bd9700b34aec3ec7d77450f3694ed7c30d1646deee7268f5cd2085c79564bf24" exitCode=6
Jul 17 11:45:29.022548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.022443    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" event={"ID":"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13","Type":"ContainerDied","Data":"bd9700b34aec3ec7d77450f3694ed7c30d1646deee7268f5cd2085c79564bf24"}
Jul 17 11:45:29.022548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.022482    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" event={"ID":"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13","Type":"ContainerStarted","Data":"4183e01437950f41c09bd7989c708b58b5210bff15af72047befabe26e371d90"}
Jul 17 11:45:29.022758 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.022735    2566 scope.go:117] "RemoveContainer" containerID="bd9700b34aec3ec7d77450f3694ed7c30d1646deee7268f5cd2085c79564bf24"
Jul 17 11:45:29.070479 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.070455    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e151536e-9a69-46cd-ac61-9b4db3c853e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-s45q2\" (UID: \"e151536e-9a69-46cd-ac61-9b4db3c853e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"
Jul 17 11:45:29.171196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.171111    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e151536e-9a69-46cd-ac61-9b4db3c853e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-s45q2\" (UID: \"e151536e-9a69-46cd-ac61-9b4db3c853e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"
Jul 17 11:45:29.171668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.171648    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e151536e-9a69-46cd-ac61-9b4db3c853e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-s45q2\" (UID: \"e151536e-9a69-46cd-ac61-9b4db3c853e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"
Jul 17 11:45:29.304839 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.304813    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"
Jul 17 11:45:29.439949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:29.439925    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"]
Jul 17 11:45:29.441589 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:29.441553    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode151536e_9a69_46cd_ac61_9b4db3c853e0.slice/crio-4877484830b7e6f5bd1dda625a43a06d5c922d3b15745b6b3448547b0b779918 WatchSource:0}: Error finding container 4877484830b7e6f5bd1dda625a43a06d5c922d3b15745b6b3448547b0b779918: Status 404 returned error can't find the container with id 4877484830b7e6f5bd1dda625a43a06d5c922d3b15745b6b3448547b0b779918
Jul 17 11:45:30.029859 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.029773    2566 generic.go:356] "Generic (PLEG): container finished" podID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerID="bdd3738881e65b7054b1c9e7b103c816ddea069f756ed7dd9b260fc776aa65d8" exitCode=6
Jul 17 11:45:30.029859 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.029836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" event={"ID":"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13","Type":"ContainerDied","Data":"bdd3738881e65b7054b1c9e7b103c816ddea069f756ed7dd9b260fc776aa65d8"}
Jul 17 11:45:30.030369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.029913    2566 scope.go:117] "RemoveContainer" containerID="bd9700b34aec3ec7d77450f3694ed7c30d1646deee7268f5cd2085c79564bf24"
Jul 17 11:45:30.030369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.030106    2566 scope.go:117] "RemoveContainer" containerID="bdd3738881e65b7054b1c9e7b103c816ddea069f756ed7dd9b260fc776aa65d8"
Jul 17 11:45:30.030369 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:30.030339    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-kzfrc_odh-ai-gateway-infra(2b3ce2e0-ccaa-4975-a7f0-6e1872339e13)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13"
Jul 17 11:45:30.031557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.031532    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" event={"ID":"e151536e-9a69-46cd-ac61-9b4db3c853e0","Type":"ContainerStarted","Data":"490b6ed063904214f136a7f8bfea6fcb9bbb757e9d3c53cd27e90d6e7b58806c"}
Jul 17 11:45:30.031664 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.031574    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" event={"ID":"e151536e-9a69-46cd-ac61-9b4db3c853e0","Type":"ContainerStarted","Data":"4877484830b7e6f5bd1dda625a43a06d5c922d3b15745b6b3448547b0b779918"}
Jul 17 11:45:30.071790 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.071737    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" podStartSLOduration=2.071699812 podStartE2EDuration="2.071699812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:45:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:45:30.067426262 +0000 UTC m=+2912.924510208" watchObservedRunningTime="2026-07-17 11:45:30.071699812 +0000 UTC m=+2912.928783772"
Jul 17 11:45:30.151250 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.151226    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"
Jul 17 11:45:30.281696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.281615    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c62ac0-771c-49c4-b542-6aad30b89b57-maas-api-service-ca\") pod \"00c62ac0-771c-49c4-b542-6aad30b89b57\" (UID: \"00c62ac0-771c-49c4-b542-6aad30b89b57\") "
Jul 17 11:45:30.281940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.281919    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00c62ac0-771c-49c4-b542-6aad30b89b57-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "00c62ac0-771c-49c4-b542-6aad30b89b57" (UID: "00c62ac0-771c-49c4-b542-6aad30b89b57"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:45:30.382785 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:30.382743    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00c62ac0-771c-49c4-b542-6aad30b89b57-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:31.038750 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.038700    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8"
Jul 17 11:45:31.038750 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.038741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8" event={"ID":"00c62ac0-771c-49c4-b542-6aad30b89b57","Type":"ContainerDied","Data":"6dafe0c0ef522cf0f013813064b03fc18f804afbe689fcb457bd3369c2705066"}
Jul 17 11:45:31.039221 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.038790    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6dafe0c0ef522cf0f013813064b03fc18f804afbe689fcb457bd3369c2705066"
Jul 17 11:45:31.040649 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.040614    2566 scope.go:117] "RemoveContainer" containerID="bdd3738881e65b7054b1c9e7b103c816ddea069f756ed7dd9b260fc776aa65d8"
Jul 17 11:45:31.040835 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:31.040813    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-kzfrc_odh-ai-gateway-infra(2b3ce2e0-ccaa-4975-a7f0-6e1872339e13)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13"
Jul 17 11:45:31.041791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.041767    2566 generic.go:356] "Generic (PLEG): container finished" podID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerID="490b6ed063904214f136a7f8bfea6fcb9bbb757e9d3c53cd27e90d6e7b58806c" exitCode=6
Jul 17 11:45:31.041902 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.041846    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" event={"ID":"e151536e-9a69-46cd-ac61-9b4db3c853e0","Type":"ContainerDied","Data":"490b6ed063904214f136a7f8bfea6fcb9bbb757e9d3c53cd27e90d6e7b58806c"}
Jul 17 11:45:31.042006 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.041993    2566 scope.go:117] "RemoveContainer" containerID="490b6ed063904214f136a7f8bfea6fcb9bbb757e9d3c53cd27e90d6e7b58806c"
Jul 17 11:45:31.693022 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.693002    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:45:31.795174 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795149    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-istio-token\") pod \"08472687-6ac3-4870-b620-d1ffa890c449\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") "
Jul 17 11:45:31.795365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795180    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-certs\") pod \"08472687-6ac3-4870-b620-d1ffa890c449\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") "
Jul 17 11:45:31.795365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795213    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-envoy\") pod \"08472687-6ac3-4870-b620-d1ffa890c449\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") "
Jul 17 11:45:31.795365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795265    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-data\") pod \"08472687-6ac3-4870-b620-d1ffa890c449\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") "
Jul 17 11:45:31.795365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795292    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-socket\") pod \"08472687-6ac3-4870-b620-d1ffa890c449\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") "
Jul 17 11:45:31.795365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795318    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08472687-6ac3-4870-b620-d1ffa890c449-istio-podinfo\") pod \"08472687-6ac3-4870-b620-d1ffa890c449\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") "
Jul 17 11:45:31.795365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795345    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08472687-6ac3-4870-b620-d1ffa890c449-istiod-ca-cert\") pod \"08472687-6ac3-4870-b620-d1ffa890c449\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") "
Jul 17 11:45:31.795694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795373    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-credential-socket\") pod \"08472687-6ac3-4870-b620-d1ffa890c449\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") "
Jul 17 11:45:31.795694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795402    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr8z9\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-kube-api-access-kr8z9\") pod \"08472687-6ac3-4870-b620-d1ffa890c449\" (UID: \"08472687-6ac3-4870-b620-d1ffa890c449\") "
Jul 17 11:45:31.795694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795475    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "08472687-6ac3-4870-b620-d1ffa890c449" (UID: "08472687-6ac3-4870-b620-d1ffa890c449"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:45:31.795694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795656    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "08472687-6ac3-4870-b620-d1ffa890c449" (UID: "08472687-6ac3-4870-b620-d1ffa890c449"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:45:31.795911 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795757    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08472687-6ac3-4870-b620-d1ffa890c449-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "08472687-6ac3-4870-b620-d1ffa890c449" (UID: "08472687-6ac3-4870-b620-d1ffa890c449"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:45:31.795911 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795777    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:31.795911 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795798    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:31.796043 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.795962    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "08472687-6ac3-4870-b620-d1ffa890c449" (UID: "08472687-6ac3-4870-b620-d1ffa890c449"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:45:31.796408 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.796383    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-data" (OuterVolumeSpecName: "istio-data") pod "08472687-6ac3-4870-b620-d1ffa890c449" (UID: "08472687-6ac3-4870-b620-d1ffa890c449"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:45:31.797662 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.797628    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-kube-api-access-kr8z9" (OuterVolumeSpecName: "kube-api-access-kr8z9") pod "08472687-6ac3-4870-b620-d1ffa890c449" (UID: "08472687-6ac3-4870-b620-d1ffa890c449"). InnerVolumeSpecName "kube-api-access-kr8z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:45:31.797796 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.797701    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08472687-6ac3-4870-b620-d1ffa890c449-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "08472687-6ac3-4870-b620-d1ffa890c449" (UID: "08472687-6ac3-4870-b620-d1ffa890c449"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:45:31.797940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.797920    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "08472687-6ac3-4870-b620-d1ffa890c449" (UID: "08472687-6ac3-4870-b620-d1ffa890c449"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:45:31.797940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.797932    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-istio-token" (OuterVolumeSpecName: "istio-token") pod "08472687-6ac3-4870-b620-d1ffa890c449" (UID: "08472687-6ac3-4870-b620-d1ffa890c449"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:45:31.896843 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.896792    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:31.896843 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.896816    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:31.896843 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.896829    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08472687-6ac3-4870-b620-d1ffa890c449-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:31.896843 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.896837    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08472687-6ac3-4870-b620-d1ffa890c449-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:31.896843 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.896845    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08472687-6ac3-4870-b620-d1ffa890c449-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:31.897054 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.896854    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr8z9\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-kube-api-access-kr8z9\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:31.897054 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:31.896862    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08472687-6ac3-4870-b620-d1ffa890c449-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:32.048955 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.048926    2566 generic.go:356] "Generic (PLEG): container finished" podID="08472687-6ac3-4870-b620-d1ffa890c449" containerID="e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e" exitCode=0
Jul 17 11:45:32.049334 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.048979    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"
Jul 17 11:45:32.049334 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.049016    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" event={"ID":"08472687-6ac3-4870-b620-d1ffa890c449","Type":"ContainerDied","Data":"e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e"}
Jul 17 11:45:32.049334 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.049086    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf" event={"ID":"08472687-6ac3-4870-b620-d1ffa890c449","Type":"ContainerDied","Data":"6e921e2b6aa98fadbb862398d7eeb49a1ff574690a5c05c2691ffbb2a4622b69"}
Jul 17 11:45:32.049334 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.049119    2566 scope.go:117] "RemoveContainer" containerID="e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e"
Jul 17 11:45:32.050940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.050917    2566 generic.go:356] "Generic (PLEG): container finished" podID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerID="43439a04613b89e53118416427ca232d5a64c409f72bd2ce964dc1a3f4608dae" exitCode=6
Jul 17 11:45:32.051048 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.050968    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" event={"ID":"e151536e-9a69-46cd-ac61-9b4db3c853e0","Type":"ContainerDied","Data":"43439a04613b89e53118416427ca232d5a64c409f72bd2ce964dc1a3f4608dae"}
Jul 17 11:45:32.051169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.051154    2566 scope.go:117] "RemoveContainer" containerID="43439a04613b89e53118416427ca232d5a64c409f72bd2ce964dc1a3f4608dae"
Jul 17 11:45:32.051328 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:32.051312    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-s45q2_odh-ai-gateway-infra(e151536e-9a69-46cd-ac61-9b4db3c853e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0"
Jul 17 11:45:32.058179 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.058164    2566 scope.go:117] "RemoveContainer" containerID="cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419"
Jul 17 11:45:32.066011 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.065997    2566 scope.go:117] "RemoveContainer" containerID="e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e"
Jul 17 11:45:32.066223 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:32.066208    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e\": container with ID starting with e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e not found: ID does not exist" containerID="e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e"
Jul 17 11:45:32.066273 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.066229    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e"} err="failed to get container status \"e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e\": rpc error: code = NotFound desc = could not find container \"e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e\": container with ID starting with e257377d4ba1f1576a66173488bdd279f0889059bf046e203b1afb979f12767e not found: ID does not exist"
Jul 17 11:45:32.066273 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.066243    2566 scope.go:117] "RemoveContainer" containerID="cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419"
Jul 17 11:45:32.066430 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:32.066412    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419\": container with ID starting with cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419 not found: ID does not exist" containerID="cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419"
Jul 17 11:45:32.066468 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.066434    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419"} err="failed to get container status \"cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419\": rpc error: code = NotFound desc = could not find container \"cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419\": container with ID starting with cce2a585b2f1e7fbd28893b138fd47cb85e14830f1019e5f3bebbd60d35b4419 not found: ID does not exist"
Jul 17 11:45:32.066468 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.066446    2566 scope.go:117] "RemoveContainer" containerID="490b6ed063904214f136a7f8bfea6fcb9bbb757e9d3c53cd27e90d6e7b58806c"
Jul 17 11:45:32.091804 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.091777    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"]
Jul 17 11:45:32.098037 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:32.098016    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-fc3287-openshift-default-86756988c-p2vnf"]
Jul 17 11:45:33.057966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:33.057936    2566 scope.go:117] "RemoveContainer" containerID="43439a04613b89e53118416427ca232d5a64c409f72bd2ce964dc1a3f4608dae"
Jul 17 11:45:33.058369 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:33.058160    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-s45q2_odh-ai-gateway-infra(e151536e-9a69-46cd-ac61-9b4db3c853e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0"
Jul 17 11:45:33.657967 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:33.657926    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08472687-6ac3-4870-b620-d1ffa890c449" path="/var/lib/kubelet/pods/08472687-6ac3-4870-b620-d1ffa890c449/volumes"
Jul 17 11:45:37.477887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:37.477852    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz"]
Jul 17 11:45:37.481020 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:37.480998    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7c5b42-29738145-qgdtz"]
Jul 17 11:45:37.657594 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:37.657564    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ae23873-c392-4119-9128-e4e633040407" path="/var/lib/kubelet/pods/4ae23873-c392-4119-9128-e4e633040407/volumes"
Jul 17 11:45:40.543923 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:40.543886    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:45:43.652418 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:43.652373    2566 scope.go:117] "RemoveContainer" containerID="43439a04613b89e53118416427ca232d5a64c409f72bd2ce964dc1a3f4608dae"
Jul 17 11:45:43.652852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:43.652584    2566 scope.go:117] "RemoveContainer" containerID="bdd3738881e65b7054b1c9e7b103c816ddea069f756ed7dd9b260fc776aa65d8"
Jul 17 11:45:44.121201 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:44.121170    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" event={"ID":"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13","Type":"ContainerStarted","Data":"cfcf9a07216354ae8ff86d4657f74618fcf6229eba219bf9449a55aa7dec8c90"}
Jul 17 11:45:44.122754 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:44.122732    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" event={"ID":"e151536e-9a69-46cd-ac61-9b4db3c853e0","Type":"ContainerStarted","Data":"fea577d3201d85a72bdb72cdac235cf71a89dcb8c74b285522e1edafeda06cf9"}
Jul 17 11:45:44.139324 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:44.139275    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" podStartSLOduration=17.139263992 podStartE2EDuration="17.139263992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:45:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:45:44.138524935 +0000 UTC m=+2926.995608881" watchObservedRunningTime="2026-07-17 11:45:44.139263992 +0000 UTC m=+2926.996347938"
Jul 17 11:45:45.130116 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.130078    2566 generic.go:356] "Generic (PLEG): container finished" podID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerID="cfcf9a07216354ae8ff86d4657f74618fcf6229eba219bf9449a55aa7dec8c90" exitCode=6
Jul 17 11:45:45.130568 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.130152    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" event={"ID":"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13","Type":"ContainerDied","Data":"cfcf9a07216354ae8ff86d4657f74618fcf6229eba219bf9449a55aa7dec8c90"}
Jul 17 11:45:45.130568 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.130220    2566 scope.go:117] "RemoveContainer" containerID="bdd3738881e65b7054b1c9e7b103c816ddea069f756ed7dd9b260fc776aa65d8"
Jul 17 11:45:45.130568 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.130460    2566 scope.go:117] "RemoveContainer" containerID="cfcf9a07216354ae8ff86d4657f74618fcf6229eba219bf9449a55aa7dec8c90"
Jul 17 11:45:45.130794 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:45.130684    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-kzfrc_odh-ai-gateway-infra(2b3ce2e0-ccaa-4975-a7f0-6e1872339e13)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13"
Jul 17 11:45:45.132178 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.132158    2566 generic.go:356] "Generic (PLEG): container finished" podID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerID="fea577d3201d85a72bdb72cdac235cf71a89dcb8c74b285522e1edafeda06cf9" exitCode=6
Jul 17 11:45:45.132275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.132197    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" event={"ID":"e151536e-9a69-46cd-ac61-9b4db3c853e0","Type":"ContainerDied","Data":"fea577d3201d85a72bdb72cdac235cf71a89dcb8c74b285522e1edafeda06cf9"}
Jul 17 11:45:45.132449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.132436    2566 scope.go:117] "RemoveContainer" containerID="fea577d3201d85a72bdb72cdac235cf71a89dcb8c74b285522e1edafeda06cf9"
Jul 17 11:45:45.132608 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:45.132590    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-s45q2_odh-ai-gateway-infra(e151536e-9a69-46cd-ac61-9b4db3c853e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0"
Jul 17 11:45:45.141228 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.141212    2566 scope.go:117] "RemoveContainer" containerID="43439a04613b89e53118416427ca232d5a64c409f72bd2ce964dc1a3f4608dae"
Jul 17 11:45:45.156830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.156810    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"]
Jul 17 11:45:45.175213 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:45.175188    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"]
Jul 17 11:45:46.296828 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:46.296804    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"
Jul 17 11:45:46.299498 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:46.299476    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"
Jul 17 11:45:46.326840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:46.326814    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e151536e-9a69-46cd-ac61-9b4db3c853e0-maas-api-service-ca\") pod \"e151536e-9a69-46cd-ac61-9b4db3c853e0\" (UID: \"e151536e-9a69-46cd-ac61-9b4db3c853e0\") "
Jul 17 11:45:46.326991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:46.326938    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3ce2e0-ccaa-4975-a7f0-6e1872339e13-maas-api-service-ca\") pod \"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13\" (UID: \"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13\") "
Jul 17 11:45:46.327218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:46.327192    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e151536e-9a69-46cd-ac61-9b4db3c853e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e151536e-9a69-46cd-ac61-9b4db3c853e0" (UID: "e151536e-9a69-46cd-ac61-9b4db3c853e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:45:46.327316 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:46.327258    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b3ce2e0-ccaa-4975-a7f0-6e1872339e13-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" (UID: "2b3ce2e0-ccaa-4975-a7f0-6e1872339e13"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:45:46.428313 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:46.428231    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e151536e-9a69-46cd-ac61-9b4db3c853e0-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:46.428313 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:46.428257    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3ce2e0-ccaa-4975-a7f0-6e1872339e13-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:45:47.147369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.147334    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc" event={"ID":"2b3ce2e0-ccaa-4975-a7f0-6e1872339e13","Type":"ContainerDied","Data":"4183e01437950f41c09bd7989c708b58b5210bff15af72047befabe26e371d90"}
Jul 17 11:45:47.147667 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.147399    2566 scope.go:117] "RemoveContainer" containerID="cfcf9a07216354ae8ff86d4657f74618fcf6229eba219bf9449a55aa7dec8c90"
Jul 17 11:45:47.147667 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.147341    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"
Jul 17 11:45:47.148838 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.148662    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"
Jul 17 11:45:47.148838 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.148666    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2" event={"ID":"e151536e-9a69-46cd-ac61-9b4db3c853e0","Type":"ContainerDied","Data":"4877484830b7e6f5bd1dda625a43a06d5c922d3b15745b6b3448547b0b779918"}
Jul 17 11:45:47.156637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.156618    2566 scope.go:117] "RemoveContainer" containerID="fea577d3201d85a72bdb72cdac235cf71a89dcb8c74b285522e1edafeda06cf9"
Jul 17 11:45:47.180180 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.180154    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"]
Jul 17 11:45:47.188676 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.188658    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-s45q2"]
Jul 17 11:45:47.205003 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.204981    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"]
Jul 17 11:45:47.215683 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.215663    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-kzfrc"]
Jul 17 11:45:47.657027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.656991    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" path="/var/lib/kubelet/pods/2b3ce2e0-ccaa-4975-a7f0-6e1872339e13/volumes"
Jul 17 11:45:47.657530 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:47.657494    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" path="/var/lib/kubelet/pods/e151536e-9a69-46cd-ac61-9b4db3c853e0/volumes"
Jul 17 11:45:53.597410 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597375    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"]
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597743    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597754    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597763    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597768    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597778    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597783    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597790    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597794    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597804    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00c62ac0-771c-49c4-b542-6aad30b89b57" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597809    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="00c62ac0-771c-49c4-b542-6aad30b89b57" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597820    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597825    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597835    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597841    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerName="revoke-keys"
Jul 17 11:45:53.597844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597849    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerName="revoke-keys"
Jul 17 11:45:53.598369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597854    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerName="revoke-keys"
Jul 17 11:45:53.598369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597910    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerName="revoke-keys"
Jul 17 11:45:53.598369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597917    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy"
Jul 17 11:45:53.598369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597923    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="00c62ac0-771c-49c4-b542-6aad30b89b57" containerName="revoke-keys"
Jul 17 11:45:53.598369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597928    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerName="revoke-keys"
Jul 17 11:45:53.598369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597933    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerName="revoke-keys"
Jul 17 11:45:53.598369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.597941    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerName="revoke-keys"
Jul 17 11:45:53.602751 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.602732    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.605629 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.605611    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c0bf60-openshift-default-dockercfg-xzwbv\""
Jul 17 11:45:53.613735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.613697    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"]
Jul 17 11:45:53.686889 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.686847    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-token\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.687054 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.686900    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqnv6\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-kube-api-access-bqnv6\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.687054 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.686967    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d163057-f4ff-4591-aa88-5f6adceaa97e-istiod-ca-cert\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.687054 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.687035    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-data\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.687188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.687069    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-envoy\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.687188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.687086    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-podinfo\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.687188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.687116    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-certs\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.687188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.687160    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-credential-socket\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.687188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.687179    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-socket\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.787683 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.787630    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqnv6\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-kube-api-access-bqnv6\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.787683 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.787691    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d163057-f4ff-4591-aa88-5f6adceaa97e-istiod-ca-cert\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.787975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.787782    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-data\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.787975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.787821    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-envoy\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.787975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.787842    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-podinfo\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.787975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.787871    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-certs\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.787975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.787906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-credential-socket\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.787975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.787932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-socket\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.788291 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.787991    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-token\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.788359 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.788332    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-data\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.788420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.788375    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-certs\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.788483 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.788413    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-credential-socket\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.788544 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.788504    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-socket\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.788544 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.788523    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d163057-f4ff-4591-aa88-5f6adceaa97e-istiod-ca-cert\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.790608 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.790580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-envoy\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.790834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.790816    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-podinfo\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.798767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.798746    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqnv6\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-kube-api-access-bqnv6\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.798944 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.798922    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-token\") pod \"e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") " pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:53.912895 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:53.912795    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:54.049553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:54.049523    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"]
Jul 17 11:45:54.052488 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:54.052457    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d163057_f4ff_4591_aa88_5f6adceaa97e.slice/crio-c465121de720536742ca388ca13c4db8632292a7457ac6623d39c52848506881 WatchSource:0}: Error finding container c465121de720536742ca388ca13c4db8632292a7457ac6623d39c52848506881: Status 404 returned error can't find the container with id c465121de720536742ca388ca13c4db8632292a7457ac6623d39c52848506881
Jul 17 11:45:54.054520 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:54.054486    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:45:54.054605 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:54.054551    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:45:54.054605 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:54.054582    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:45:54.189792 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:54.189679    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" event={"ID":"5d163057-f4ff-4591-aa88-5f6adceaa97e","Type":"ContainerStarted","Data":"a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b"}
Jul 17 11:45:54.189792 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:54.189759    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" event={"ID":"5d163057-f4ff-4591-aa88-5f6adceaa97e","Type":"ContainerStarted","Data":"c465121de720536742ca388ca13c4db8632292a7457ac6623d39c52848506881"}
Jul 17 11:45:54.214341 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:54.214287    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" podStartSLOduration=1.214269419 podStartE2EDuration="1.214269419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:45:54.212476985 +0000 UTC m=+2937.069560942" watchObservedRunningTime="2026-07-17 11:45:54.214269419 +0000 UTC m=+2937.071353364"
Jul 17 11:45:54.913767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:54.913736    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:55.757518 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.757414    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"]
Jul 17 11:45:55.759005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.758854    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerName="revoke-keys"
Jul 17 11:45:55.759005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.758936    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerName="revoke-keys"
Jul 17 11:45:55.759548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.759508    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b3ce2e0-ccaa-4975-a7f0-6e1872339e13" containerName="revoke-keys"
Jul 17 11:45:55.759714 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.759666    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="08472687-6ac3-4870-b620-d1ffa890c449" containerName="istio-proxy"
Jul 17 11:45:55.759862 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.759836    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e151536e-9a69-46cd-ac61-9b4db3c853e0" containerName="revoke-keys"
Jul 17 11:45:55.764305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.764276    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"
Jul 17 11:45:55.767477 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.767450    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:45:55.767631 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.767468    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:45:55.768307 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.768284    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"]
Jul 17 11:45:55.804559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.804532    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093f3b0d-e062-4ff0-8aea-4d455f3b16ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-52ftv\" (UID: \"093f3b0d-e062-4ff0-8aea-4d455f3b16ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"
Jul 17 11:45:55.905881 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.905847    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093f3b0d-e062-4ff0-8aea-4d455f3b16ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-52ftv\" (UID: \"093f3b0d-e062-4ff0-8aea-4d455f3b16ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"
Jul 17 11:45:55.906493 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.906475    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093f3b0d-e062-4ff0-8aea-4d455f3b16ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-52ftv\" (UID: \"093f3b0d-e062-4ff0-8aea-4d455f3b16ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"
Jul 17 11:45:55.918572 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:55.918551    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:56.076223 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:56.076191    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"
Jul 17 11:45:56.200748 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:56.200716    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:56.201777 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:56.201756    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:45:56.209029 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:56.209006    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"]
Jul 17 11:45:56.210688 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:45:56.210665    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093f3b0d_e062_4ff0_8aea_4d455f3b16ad.slice/crio-a037ae5eeec3cac39e5d802c924c6891bd6c0f70fa48aee2b84bb32971709fdf WatchSource:0}: Error finding container a037ae5eeec3cac39e5d802c924c6891bd6c0f70fa48aee2b84bb32971709fdf: Status 404 returned error can't find the container with id a037ae5eeec3cac39e5d802c924c6891bd6c0f70fa48aee2b84bb32971709fdf
Jul 17 11:45:57.209127 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:57.209039    2566 generic.go:356] "Generic (PLEG): container finished" podID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerID="b0686175240eead7e3850455dbc5d8ad5d6bc19f4577de8ac8e96fa9589aa893" exitCode=6
Jul 17 11:45:57.209610 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:57.209133    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" event={"ID":"093f3b0d-e062-4ff0-8aea-4d455f3b16ad","Type":"ContainerDied","Data":"b0686175240eead7e3850455dbc5d8ad5d6bc19f4577de8ac8e96fa9589aa893"}
Jul 17 11:45:57.209610 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:57.209197    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" event={"ID":"093f3b0d-e062-4ff0-8aea-4d455f3b16ad","Type":"ContainerStarted","Data":"a037ae5eeec3cac39e5d802c924c6891bd6c0f70fa48aee2b84bb32971709fdf"}
Jul 17 11:45:57.209610 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:57.209379    2566 scope.go:117] "RemoveContainer" containerID="b0686175240eead7e3850455dbc5d8ad5d6bc19f4577de8ac8e96fa9589aa893"
Jul 17 11:45:58.216032 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:58.216004    2566 generic.go:356] "Generic (PLEG): container finished" podID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerID="8319f593dff91895f94e08ce96fe8c5f0d71b3dbdc378904e959857f31e49f58" exitCode=6
Jul 17 11:45:58.216381 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:58.216089    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" event={"ID":"093f3b0d-e062-4ff0-8aea-4d455f3b16ad","Type":"ContainerDied","Data":"8319f593dff91895f94e08ce96fe8c5f0d71b3dbdc378904e959857f31e49f58"}
Jul 17 11:45:58.216381 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:58.216161    2566 scope.go:117] "RemoveContainer" containerID="b0686175240eead7e3850455dbc5d8ad5d6bc19f4577de8ac8e96fa9589aa893"
Jul 17 11:45:58.216381 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:58.216359    2566 scope.go:117] "RemoveContainer" containerID="8319f593dff91895f94e08ce96fe8c5f0d71b3dbdc378904e959857f31e49f58"
Jul 17 11:45:58.216620 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:58.216600    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-52ftv_odh-ai-gateway-infra(093f3b0d-e062-4ff0-8aea-4d455f3b16ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad"
Jul 17 11:45:59.222548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:45:59.222523    2566 scope.go:117] "RemoveContainer" containerID="8319f593dff91895f94e08ce96fe8c5f0d71b3dbdc378904e959857f31e49f58"
Jul 17 11:45:59.222965 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:45:59.222746    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-52ftv_odh-ai-gateway-infra(093f3b0d-e062-4ff0-8aea-4d455f3b16ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad"
Jul 17 11:46:01.498727 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.496155    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"]
Jul 17 11:46:01.506808 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.505453    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.509382 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.509202    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a50d5b-openshift-default-dockercfg-znsxl\""
Jul 17 11:46:01.512094 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.512069    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"]
Jul 17 11:46:01.554890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.554861    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-socket\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.555009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.554893    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-token\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.555009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.554909    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-credential-socket\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.555009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.554924    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-data\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.555009 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.554940    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwn2q\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-kube-api-access-zwn2q\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.555143 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.555026    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-envoy\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.555143 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.555060    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-certs\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.555205 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.555169    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istiod-ca-cert\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.555205 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.555196    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-podinfo\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656151 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istiod-ca-cert\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-podinfo\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656216    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-socket\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-token\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656259    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-credential-socket\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-data\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656543 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwn2q\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-kube-api-access-zwn2q\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656543 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656356    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-envoy\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656657 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656560    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-certs\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656827 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656807    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-socket\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.656908 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656875    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-data\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.657017 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.656993    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-certs\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.657131 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.657108    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-credential-socket\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.657444 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.657422    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istiod-ca-cert\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.658613 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.658590    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-envoy\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.658791 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.658775    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-podinfo\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.665451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.665425    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwn2q\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-kube-api-access-zwn2q\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.665561 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.665545    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-token\") pod \"e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") " pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.820815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.820734    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:01.960834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.960807    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"]
Jul 17 11:46:01.962561 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:46:01.962528    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8b6296_1cd3_4010_ad98_9156ec28e3a5.slice/crio-0c81401045d7a7747be8f5312e0b68d34fecb1982e6bad5f078e0a642eed2bd2 WatchSource:0}: Error finding container 0c81401045d7a7747be8f5312e0b68d34fecb1982e6bad5f078e0a642eed2bd2: Status 404 returned error can't find the container with id 0c81401045d7a7747be8f5312e0b68d34fecb1982e6bad5f078e0a642eed2bd2
Jul 17 11:46:01.964721 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.964677    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:46:01.964824 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.964764    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:46:01.964824 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:01.964811    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:46:02.239765 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:02.239732    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" event={"ID":"6b8b6296-1cd3-4010-ad98-9156ec28e3a5","Type":"ContainerStarted","Data":"cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07"}
Jul 17 11:46:02.239955 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:02.239783    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" event={"ID":"6b8b6296-1cd3-4010-ad98-9156ec28e3a5","Type":"ContainerStarted","Data":"0c81401045d7a7747be8f5312e0b68d34fecb1982e6bad5f078e0a642eed2bd2"}
Jul 17 11:46:02.261051 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:02.261001    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" podStartSLOduration=1.260988075 podStartE2EDuration="1.260988075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:46:02.259696441 +0000 UTC m=+2945.116780410" watchObservedRunningTime="2026-07-17 11:46:02.260988075 +0000 UTC m=+2945.118072020"
Jul 17 11:46:02.821832 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:02.821798    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:03.825857 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:03.825827    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:04.036403 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:04.036370    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:46:04.251140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:04.251063    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:04.251976 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:04.251957    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:12.131991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:12.130976    2566 patch_prober.go:28] interesting pod/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": read tcp 10.132.0.2:46264->10.132.0.110:15021: read: connection reset by peer" start-of-body=
Jul 17 11:46:12.131991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:12.131061    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": read tcp 10.132.0.2:46264->10.132.0.110:15021: read: connection reset by peer"
Jul 17 11:46:12.297246 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:12.297164    2566 generic.go:356] "Generic (PLEG): container finished" podID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerID="a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b" exitCode=0
Jul 17 11:46:12.297408 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:12.297236    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" event={"ID":"5d163057-f4ff-4591-aa88-5f6adceaa97e","Type":"ContainerDied","Data":"a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b"}
Jul 17 11:46:12.297642 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:12.297628    2566 scope.go:117] "RemoveContainer" containerID="a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b"
Jul 17 11:46:12.298640 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:12.298609    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:46:12.298774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:12.298675    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:46:12.298774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:12.298727    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:46:13.304083 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:13.304053    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" event={"ID":"5d163057-f4ff-4591-aa88-5f6adceaa97e","Type":"ContainerStarted","Data":"0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a"}
Jul 17 11:46:13.913099 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:13.913059    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:46:13.913321 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:13.913297    2566 patch_prober.go:28] interesting pod/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Jul 17 11:46:13.913381 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:13.913347    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Jul 17 11:46:14.652539 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:14.652506    2566 scope.go:117] "RemoveContainer" containerID="8319f593dff91895f94e08ce96fe8c5f0d71b3dbdc378904e959857f31e49f58"
Jul 17 11:46:14.913809 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:14.913699    2566 patch_prober.go:28] interesting pod/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Jul 17 11:46:14.913809 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:14.913774    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Jul 17 11:46:15.315664 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:15.315575    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" event={"ID":"093f3b0d-e062-4ff0-8aea-4d455f3b16ad","Type":"ContainerStarted","Data":"a5e14f059e0336b7544cbb78df4b7f3002e0d9b796842aa67afde2478779d47e"}
Jul 17 11:46:15.914104 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:15.914072    2566 patch_prober.go:28] interesting pod/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Jul 17 11:46:15.914558 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:15.914279    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Jul 17 11:46:16.322239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:16.322213    2566 generic.go:356] "Generic (PLEG): container finished" podID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerID="a5e14f059e0336b7544cbb78df4b7f3002e0d9b796842aa67afde2478779d47e" exitCode=6
Jul 17 11:46:16.322388 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:16.322269    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" event={"ID":"093f3b0d-e062-4ff0-8aea-4d455f3b16ad","Type":"ContainerDied","Data":"a5e14f059e0336b7544cbb78df4b7f3002e0d9b796842aa67afde2478779d47e"}
Jul 17 11:46:16.322388 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:16.322310    2566 scope.go:117] "RemoveContainer" containerID="8319f593dff91895f94e08ce96fe8c5f0d71b3dbdc378904e959857f31e49f58"
Jul 17 11:46:16.322603 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:16.322591    2566 scope.go:117] "RemoveContainer" containerID="a5e14f059e0336b7544cbb78df4b7f3002e0d9b796842aa67afde2478779d47e"
Jul 17 11:46:16.322844 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:46:16.322821    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-52ftv_odh-ai-gateway-infra(093f3b0d-e062-4ff0-8aea-4d455f3b16ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad"
Jul 17 11:46:16.344172 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:16.344098    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" podStartSLOduration=21.344075255 podStartE2EDuration="21.344075255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:46:15.339145112 +0000 UTC m=+2958.196229059" watchObservedRunningTime="2026-07-17 11:46:16.344075255 +0000 UTC m=+2959.201159203"
Jul 17 11:46:16.346844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:16.346813    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"]
Jul 17 11:46:16.913421 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:16.913383    2566 patch_prober.go:28] interesting pod/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Jul 17 11:46:16.913616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:16.913444    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Jul 17 11:46:17.510186 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:17.510161    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"
Jul 17 11:46:17.605087 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:17.604991    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093f3b0d-e062-4ff0-8aea-4d455f3b16ad-maas-api-service-ca\") pod \"093f3b0d-e062-4ff0-8aea-4d455f3b16ad\" (UID: \"093f3b0d-e062-4ff0-8aea-4d455f3b16ad\") "
Jul 17 11:46:17.605354 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:17.605331    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/093f3b0d-e062-4ff0-8aea-4d455f3b16ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "093f3b0d-e062-4ff0-8aea-4d455f3b16ad" (UID: "093f3b0d-e062-4ff0-8aea-4d455f3b16ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:46:17.706169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:17.706137    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093f3b0d-e062-4ff0-8aea-4d455f3b16ad-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:17.917580 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:17.917512    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:46:17.917928 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:17.917912    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:46:17.918510 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:17.918494    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:46:18.335840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:18.335802    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv" event={"ID":"093f3b0d-e062-4ff0-8aea-4d455f3b16ad","Type":"ContainerDied","Data":"a037ae5eeec3cac39e5d802c924c6891bd6c0f70fa48aee2b84bb32971709fdf"}
Jul 17 11:46:18.335840 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:18.335827    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"
Jul 17 11:46:18.336374 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:18.335860    2566 scope.go:117] "RemoveContainer" containerID="a5e14f059e0336b7544cbb78df4b7f3002e0d9b796842aa67afde2478779d47e"
Jul 17 11:46:18.372126 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:18.372096    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"]
Jul 17 11:46:18.385950 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:18.385921    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-52ftv"]
Jul 17 11:46:19.658942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:19.658912    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" path="/var/lib/kubelet/pods/093f3b0d-e062-4ff0-8aea-4d455f3b16ad/volumes"
Jul 17 11:46:20.291904 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:20.291818    2566 patch_prober.go:28] interesting pod/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 11:46:20.291904 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:20.291884    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 11:46:20.546133 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:20.546054    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:46:21.353033 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:21.353002    2566 generic.go:356] "Generic (PLEG): container finished" podID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerID="cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07" exitCode=0
Jul 17 11:46:21.353486 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:21.353076    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" event={"ID":"6b8b6296-1cd3-4010-ad98-9156ec28e3a5","Type":"ContainerDied","Data":"cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07"}
Jul 17 11:46:21.353536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:21.353490    2566 scope.go:117] "RemoveContainer" containerID="cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07"
Jul 17 11:46:21.354515 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:21.354487    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:46:21.354608 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:21.354551    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:46:21.354608 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:21.354584    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:46:22.360575 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:22.360540    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" event={"ID":"6b8b6296-1cd3-4010-ad98-9156ec28e3a5","Type":"ContainerStarted","Data":"893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56"}
Jul 17 11:46:22.821384 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:22.821288    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:22.821557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:22.821498    2566 patch_prober.go:28] interesting pod/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Jul 17 11:46:22.821631 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:22.821547    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Jul 17 11:46:23.821857 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:23.821826    2566 patch_prober.go:28] interesting pod/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Jul 17 11:46:23.822270 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:23.821878    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Jul 17 11:46:24.821832 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:24.821796    2566 patch_prober.go:28] interesting pod/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Jul 17 11:46:24.822044 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:24.821859    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Jul 17 11:46:25.821177 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:25.821146    2566 patch_prober.go:28] interesting pod/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Jul 17 11:46:25.821354 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:25.821195    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Jul 17 11:46:26.824692 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:26.824666    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:26.825056 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:26.824953    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:26.825543 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:26.825525    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:46:30.537092 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:30.537058    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:46:42.825891 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:42.825850    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"]
Jul 17 11:46:42.826420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:42.826108    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy" containerID="cri-o://0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a" gracePeriod=30
Jul 17 11:46:47.919588 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:47.919549    2566 patch_prober.go:28] interesting pod/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Jul 17 11:46:47.920040 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:47.919632    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Jul 17 11:46:48.083913 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.083890    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:46:48.182177 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182076    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-data\") pod \"5d163057-f4ff-4591-aa88-5f6adceaa97e\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") "
Jul 17 11:46:48.182177 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182130    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-certs\") pod \"5d163057-f4ff-4591-aa88-5f6adceaa97e\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") "
Jul 17 11:46:48.182177 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182178    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-podinfo\") pod \"5d163057-f4ff-4591-aa88-5f6adceaa97e\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") "
Jul 17 11:46:48.182449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182198    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqnv6\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-kube-api-access-bqnv6\") pod \"5d163057-f4ff-4591-aa88-5f6adceaa97e\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") "
Jul 17 11:46:48.182449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182244    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d163057-f4ff-4591-aa88-5f6adceaa97e-istiod-ca-cert\") pod \"5d163057-f4ff-4591-aa88-5f6adceaa97e\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") "
Jul 17 11:46:48.182449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182278    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-socket\") pod \"5d163057-f4ff-4591-aa88-5f6adceaa97e\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") "
Jul 17 11:46:48.182449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182314    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-envoy\") pod \"5d163057-f4ff-4591-aa88-5f6adceaa97e\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") "
Jul 17 11:46:48.182449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182353    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-token\") pod \"5d163057-f4ff-4591-aa88-5f6adceaa97e\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") "
Jul 17 11:46:48.182449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182429    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-credential-socket\") pod \"5d163057-f4ff-4591-aa88-5f6adceaa97e\" (UID: \"5d163057-f4ff-4591-aa88-5f6adceaa97e\") "
Jul 17 11:46:48.182868 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182471    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5d163057-f4ff-4591-aa88-5f6adceaa97e" (UID: "5d163057-f4ff-4591-aa88-5f6adceaa97e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:46:48.182868 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182735    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5d163057-f4ff-4591-aa88-5f6adceaa97e" (UID: "5d163057-f4ff-4591-aa88-5f6adceaa97e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:46:48.182868 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182770    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:48.183030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182957    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5d163057-f4ff-4591-aa88-5f6adceaa97e" (UID: "5d163057-f4ff-4591-aa88-5f6adceaa97e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:46:48.183030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.182980    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d163057-f4ff-4591-aa88-5f6adceaa97e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5d163057-f4ff-4591-aa88-5f6adceaa97e" (UID: "5d163057-f4ff-4591-aa88-5f6adceaa97e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:46:48.183030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.183022    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5d163057-f4ff-4591-aa88-5f6adceaa97e" (UID: "5d163057-f4ff-4591-aa88-5f6adceaa97e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:46:48.184519 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.184492    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-kube-api-access-bqnv6" (OuterVolumeSpecName: "kube-api-access-bqnv6") pod "5d163057-f4ff-4591-aa88-5f6adceaa97e" (UID: "5d163057-f4ff-4591-aa88-5f6adceaa97e"). InnerVolumeSpecName "kube-api-access-bqnv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:46:48.184642 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.184582    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5d163057-f4ff-4591-aa88-5f6adceaa97e" (UID: "5d163057-f4ff-4591-aa88-5f6adceaa97e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:46:48.184955 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.184940    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5d163057-f4ff-4591-aa88-5f6adceaa97e" (UID: "5d163057-f4ff-4591-aa88-5f6adceaa97e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:46:48.185031 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.185016    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5d163057-f4ff-4591-aa88-5f6adceaa97e" (UID: "5d163057-f4ff-4591-aa88-5f6adceaa97e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:46:48.283966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.283925    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:48.283966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.283960    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqnv6\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-kube-api-access-bqnv6\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:48.283966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.283971    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d163057-f4ff-4591-aa88-5f6adceaa97e-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:48.283966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.283979    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:48.284215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.283988    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:48.284215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.283996    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:48.284215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.284004    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:48.284215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.284011    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d163057-f4ff-4591-aa88-5f6adceaa97e-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:46:48.496141 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.496061    2566 generic.go:356] "Generic (PLEG): container finished" podID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerID="0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a" exitCode=0
Jul 17 11:46:48.496141 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.496113    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"
Jul 17 11:46:48.496333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.496139    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" event={"ID":"5d163057-f4ff-4591-aa88-5f6adceaa97e","Type":"ContainerDied","Data":"0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a"}
Jul 17 11:46:48.496333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.496193    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v" event={"ID":"5d163057-f4ff-4591-aa88-5f6adceaa97e","Type":"ContainerDied","Data":"c465121de720536742ca388ca13c4db8632292a7457ac6623d39c52848506881"}
Jul 17 11:46:48.496333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.496214    2566 scope.go:117] "RemoveContainer" containerID="0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a"
Jul 17 11:46:48.505353 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.505335    2566 scope.go:117] "RemoveContainer" containerID="a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b"
Jul 17 11:46:48.515284 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.515268    2566 scope.go:117] "RemoveContainer" containerID="0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a"
Jul 17 11:46:48.515526 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:46:48.515507    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a\": container with ID starting with 0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a not found: ID does not exist" containerID="0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a"
Jul 17 11:46:48.515591 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.515534    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a"} err="failed to get container status \"0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a\": rpc error: code = NotFound desc = could not find container \"0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a\": container with ID starting with 0a0588b17e97716174efaf9c3bc40103587ea70e4e145eb8f0a9ecf17a44af8a not found: ID does not exist"
Jul 17 11:46:48.515591 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.515550    2566 scope.go:117] "RemoveContainer" containerID="a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b"
Jul 17 11:46:48.515807 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:46:48.515789    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b\": container with ID starting with a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b not found: ID does not exist" containerID="a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b"
Jul 17 11:46:48.515856 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.515812    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b"} err="failed to get container status \"a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b\": rpc error: code = NotFound desc = could not find container \"a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b\": container with ID starting with a9005f077de9d4d47a34005bfe3237a334a90ee0550d0986251460344dc9398b not found: ID does not exist"
Jul 17 11:46:48.522339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.522308    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"]
Jul 17 11:46:48.524822 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:48.524794    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c0bf60-openshift-default-69c66b7c5c-cg54v"]
Jul 17 11:46:49.657983 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:49.657951    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" path="/var/lib/kubelet/pods/5d163057-f4ff-4591-aa88-5f6adceaa97e/volumes"
Jul 17 11:46:55.803293 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803264    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"]
Jul 17 11:46:55.803697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803664    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy"
Jul 17 11:46:55.803697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803676    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy"
Jul 17 11:46:55.803697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803693    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerName="revoke-keys"
Jul 17 11:46:55.803697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803698    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerName="revoke-keys"
Jul 17 11:46:55.803855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803723    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy"
Jul 17 11:46:55.803855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803729    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy"
Jul 17 11:46:55.803855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803736    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerName="revoke-keys"
Jul 17 11:46:55.803855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803741    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerName="revoke-keys"
Jul 17 11:46:55.803855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803806    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerName="revoke-keys"
Jul 17 11:46:55.803855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803815    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy"
Jul 17 11:46:55.803855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803822    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerName="revoke-keys"
Jul 17 11:46:55.803855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803827    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerName="revoke-keys"
Jul 17 11:46:55.803855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.803834    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d163057-f4ff-4591-aa88-5f6adceaa97e" containerName="istio-proxy"
Jul 17 11:46:55.808481 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.808465    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"
Jul 17 11:46:55.811365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.811342    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:46:55.811482 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.811334    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:46:55.817134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.817107    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"]
Jul 17 11:46:55.854541 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.854512    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20fd72a-9073-4ce4-94d0-d4c2477057f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-r48vg\" (UID: \"e20fd72a-9073-4ce4-94d0-d4c2477057f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"
Jul 17 11:46:55.955164 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.955125    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20fd72a-9073-4ce4-94d0-d4c2477057f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-r48vg\" (UID: \"e20fd72a-9073-4ce4-94d0-d4c2477057f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"
Jul 17 11:46:55.955820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:55.955794    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20fd72a-9073-4ce4-94d0-d4c2477057f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-r48vg\" (UID: \"e20fd72a-9073-4ce4-94d0-d4c2477057f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"
Jul 17 11:46:56.120622 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:56.120592    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"
Jul 17 11:46:56.255338 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:56.255313    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"]
Jul 17 11:46:56.257446 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:46:56.257419    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20fd72a_9073_4ce4_94d0_d4c2477057f9.slice/crio-acceffd18acc03b88db71460777224b4985b15966ef83d16659cf6815e20f3dc WatchSource:0}: Error finding container acceffd18acc03b88db71460777224b4985b15966ef83d16659cf6815e20f3dc: Status 404 returned error can't find the container with id acceffd18acc03b88db71460777224b4985b15966ef83d16659cf6815e20f3dc
Jul 17 11:46:56.548793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:56.548761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" event={"ID":"e20fd72a-9073-4ce4-94d0-d4c2477057f9","Type":"ContainerStarted","Data":"4543a446f273086f971475b0ba678c425191a0787f5f4d3e16e6f92040d0e9d2"}
Jul 17 11:46:56.548923 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:56.548825    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" event={"ID":"e20fd72a-9073-4ce4-94d0-d4c2477057f9","Type":"ContainerStarted","Data":"acceffd18acc03b88db71460777224b4985b15966ef83d16659cf6815e20f3dc"}
Jul 17 11:46:56.567337 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:56.567290    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" podStartSLOduration=1.567275277 podStartE2EDuration="1.567275277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:46:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:46:56.566344709 +0000 UTC m=+2999.423428657" watchObservedRunningTime="2026-07-17 11:46:56.567275277 +0000 UTC m=+2999.424359222"
Jul 17 11:46:56.844043 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:56.844018    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:46:57.561315 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:57.561282    2566 generic.go:356] "Generic (PLEG): container finished" podID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerID="4543a446f273086f971475b0ba678c425191a0787f5f4d3e16e6f92040d0e9d2" exitCode=6
Jul 17 11:46:57.561504 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:57.561348    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" event={"ID":"e20fd72a-9073-4ce4-94d0-d4c2477057f9","Type":"ContainerDied","Data":"4543a446f273086f971475b0ba678c425191a0787f5f4d3e16e6f92040d0e9d2"}
Jul 17 11:46:57.561554 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:57.561539    2566 scope.go:117] "RemoveContainer" containerID="4543a446f273086f971475b0ba678c425191a0787f5f4d3e16e6f92040d0e9d2"
Jul 17 11:46:57.878962 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:57.878938    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:46:57.885201 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:57.885181    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:46:57.889803 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:57.889782    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:46:57.892844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:57.892826    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:46:58.568443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:58.568353    2566 generic.go:356] "Generic (PLEG): container finished" podID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerID="308cc87e249b5a39d5d9189cbc9e3b2fd7a0268850347a822f3edc4e62cac8ad" exitCode=6
Jul 17 11:46:58.568443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:58.568410    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" event={"ID":"e20fd72a-9073-4ce4-94d0-d4c2477057f9","Type":"ContainerDied","Data":"308cc87e249b5a39d5d9189cbc9e3b2fd7a0268850347a822f3edc4e62cac8ad"}
Jul 17 11:46:58.568757 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:58.568470    2566 scope.go:117] "RemoveContainer" containerID="4543a446f273086f971475b0ba678c425191a0787f5f4d3e16e6f92040d0e9d2"
Jul 17 11:46:58.568757 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:58.568658    2566 scope.go:117] "RemoveContainer" containerID="308cc87e249b5a39d5d9189cbc9e3b2fd7a0268850347a822f3edc4e62cac8ad"
Jul 17 11:46:58.568944 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:46:58.568923    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-r48vg_odh-ai-gateway-infra(e20fd72a-9073-4ce4-94d0-d4c2477057f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9"
Jul 17 11:46:59.576228 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:46:59.576206    2566 scope.go:117] "RemoveContainer" containerID="308cc87e249b5a39d5d9189cbc9e3b2fd7a0268850347a822f3edc4e62cac8ad"
Jul 17 11:46:59.576589 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:46:59.576404    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-r48vg_odh-ai-gateway-infra(e20fd72a-9073-4ce4-94d0-d4c2477057f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9"
Jul 17 11:47:07.997892 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:07.997850    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"]
Jul 17 11:47:07.998392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:07.998263    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerName="revoke-keys"
Jul 17 11:47:07.998392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:07.998277    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="093f3b0d-e062-4ff0-8aea-4d455f3b16ad" containerName="revoke-keys"
Jul 17 11:47:08.002576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:08.002559    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"
Jul 17 11:47:08.009524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:08.009495    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"]
Jul 17 11:47:08.068380 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:08.068347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5421a860-d1cb-4fc1-b2d2-29b4fa6974f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-dk4xj\" (UID: \"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"
Jul 17 11:47:08.169847 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:08.169793    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5421a860-d1cb-4fc1-b2d2-29b4fa6974f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-dk4xj\" (UID: \"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"
Jul 17 11:47:08.170412 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:08.170391    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5421a860-d1cb-4fc1-b2d2-29b4fa6974f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-dk4xj\" (UID: \"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"
Jul 17 11:47:08.314968 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:08.314884    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"
Jul 17 11:47:08.454676 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:08.454639    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"]
Jul 17 11:47:08.456097 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:47:08.456072    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5421a860_d1cb_4fc1_b2d2_29b4fa6974f7.slice/crio-95760b416bbfceb34825d61438f3e85fad1a96c10042cda4dee87d38212fac79 WatchSource:0}: Error finding container 95760b416bbfceb34825d61438f3e85fad1a96c10042cda4dee87d38212fac79: Status 404 returned error can't find the container with id 95760b416bbfceb34825d61438f3e85fad1a96c10042cda4dee87d38212fac79
Jul 17 11:47:08.620654 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:08.620615    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj" event={"ID":"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7","Type":"ContainerStarted","Data":"95760b416bbfceb34825d61438f3e85fad1a96c10042cda4dee87d38212fac79"}
Jul 17 11:47:09.099866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:09.099828    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"]
Jul 17 11:47:09.100349 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:09.100043    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy" containerID="cri-o://893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56" gracePeriod=30
Jul 17 11:47:09.628098 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:09.628063    2566 generic.go:356] "Generic (PLEG): container finished" podID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerID="cd394f6ae3d5932a94c3839c3e49d839a7874c22dae5a75fc3f28a659da5644a" exitCode=6
Jul 17 11:47:09.628290 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:09.628156    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj" event={"ID":"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7","Type":"ContainerDied","Data":"cd394f6ae3d5932a94c3839c3e49d839a7874c22dae5a75fc3f28a659da5644a"}
Jul 17 11:47:09.628350 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:09.628322    2566 scope.go:117] "RemoveContainer" containerID="cd394f6ae3d5932a94c3839c3e49d839a7874c22dae5a75fc3f28a659da5644a"
Jul 17 11:47:10.636414 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:10.636334    2566 generic.go:356] "Generic (PLEG): container finished" podID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerID="7750fbe1a0aee533508ea7b0cfb0f16c92d23e7aa28da0790f0242b680419189" exitCode=6
Jul 17 11:47:10.636414 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:10.636402    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj" event={"ID":"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7","Type":"ContainerDied","Data":"7750fbe1a0aee533508ea7b0cfb0f16c92d23e7aa28da0790f0242b680419189"}
Jul 17 11:47:10.636886 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:10.636460    2566 scope.go:117] "RemoveContainer" containerID="cd394f6ae3d5932a94c3839c3e49d839a7874c22dae5a75fc3f28a659da5644a"
Jul 17 11:47:10.636886 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:10.636667    2566 scope.go:117] "RemoveContainer" containerID="7750fbe1a0aee533508ea7b0cfb0f16c92d23e7aa28da0790f0242b680419189"
Jul 17 11:47:10.636974 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:10.636914    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-dk4xj_odh-ai-gateway-infra(5421a860-d1cb-4fc1-b2d2-29b4fa6974f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7"
Jul 17 11:47:11.643539 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:11.643511    2566 scope.go:117] "RemoveContainer" containerID="7750fbe1a0aee533508ea7b0cfb0f16c92d23e7aa28da0790f0242b680419189"
Jul 17 11:47:11.643940 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:11.643695    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-dk4xj_odh-ai-gateway-infra(5421a860-d1cb-4fc1-b2d2-29b4fa6974f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7"
Jul 17 11:47:11.827871 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:11.827835    2566 patch_prober.go:28] interesting pod/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 11:47:11.828070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:11.827901    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 11:47:14.358377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.358350    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:47:14.526431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526345    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-data\") pod \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") "
Jul 17 11:47:14.526431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526395    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istiod-ca-cert\") pod \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") "
Jul 17 11:47:14.526431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526429    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-token\") pod \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") "
Jul 17 11:47:14.526739 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526465    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-socket\") pod \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") "
Jul 17 11:47:14.526739 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526545    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-certs\") pod \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") "
Jul 17 11:47:14.526739 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526571    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-envoy\") pod \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") "
Jul 17 11:47:14.526739 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526598    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwn2q\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-kube-api-access-zwn2q\") pod \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") "
Jul 17 11:47:14.526739 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526620    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-podinfo\") pod \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") "
Jul 17 11:47:14.526739 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526690    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-credential-socket\") pod \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\" (UID: \"6b8b6296-1cd3-4010-ad98-9156ec28e3a5\") "
Jul 17 11:47:14.527065 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526837    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b8b6296-1cd3-4010-ad98-9156ec28e3a5" (UID: "6b8b6296-1cd3-4010-ad98-9156ec28e3a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:47:14.527065 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.526852    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b8b6296-1cd3-4010-ad98-9156ec28e3a5" (UID: "6b8b6296-1cd3-4010-ad98-9156ec28e3a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:47:14.527065 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.527016    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:14.527065 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.527039    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:14.527065 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.527045    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b8b6296-1cd3-4010-ad98-9156ec28e3a5" (UID: "6b8b6296-1cd3-4010-ad98-9156ec28e3a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:47:14.527298 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.527207    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b8b6296-1cd3-4010-ad98-9156ec28e3a5" (UID: "6b8b6296-1cd3-4010-ad98-9156ec28e3a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:47:14.527364 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.527323    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b8b6296-1cd3-4010-ad98-9156ec28e3a5" (UID: "6b8b6296-1cd3-4010-ad98-9156ec28e3a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:47:14.528949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.528928    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b8b6296-1cd3-4010-ad98-9156ec28e3a5" (UID: "6b8b6296-1cd3-4010-ad98-9156ec28e3a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:47:14.529143 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.529127    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b8b6296-1cd3-4010-ad98-9156ec28e3a5" (UID: "6b8b6296-1cd3-4010-ad98-9156ec28e3a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:47:14.529314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.529293    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b8b6296-1cd3-4010-ad98-9156ec28e3a5" (UID: "6b8b6296-1cd3-4010-ad98-9156ec28e3a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:47:14.529425 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.529405    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-kube-api-access-zwn2q" (OuterVolumeSpecName: "kube-api-access-zwn2q") pod "6b8b6296-1cd3-4010-ad98-9156ec28e3a5" (UID: "6b8b6296-1cd3-4010-ad98-9156ec28e3a5"). InnerVolumeSpecName "kube-api-access-zwn2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:47:14.628367 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.628329    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:14.628367 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.628361    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:14.628367 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.628374    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:14.628614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.628384    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:14.628614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.628395    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:14.628614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.628408    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwn2q\" (UniqueName: \"kubernetes.io/projected/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-kube-api-access-zwn2q\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:14.628614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.628419    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b8b6296-1cd3-4010-ad98-9156ec28e3a5-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:14.651493 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.651461    2566 scope.go:117] "RemoveContainer" containerID="308cc87e249b5a39d5d9189cbc9e3b2fd7a0268850347a822f3edc4e62cac8ad"
Jul 17 11:47:14.659765 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.659732    2566 generic.go:356] "Generic (PLEG): container finished" podID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerID="893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56" exitCode=0
Jul 17 11:47:14.659863 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.659789    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"
Jul 17 11:47:14.659863 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.659829    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" event={"ID":"6b8b6296-1cd3-4010-ad98-9156ec28e3a5","Type":"ContainerDied","Data":"893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56"}
Jul 17 11:47:14.659937 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.659888    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj" event={"ID":"6b8b6296-1cd3-4010-ad98-9156ec28e3a5","Type":"ContainerDied","Data":"0c81401045d7a7747be8f5312e0b68d34fecb1982e6bad5f078e0a642eed2bd2"}
Jul 17 11:47:14.659937 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.659912    2566 scope.go:117] "RemoveContainer" containerID="893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56"
Jul 17 11:47:14.669305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.669290    2566 scope.go:117] "RemoveContainer" containerID="cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07"
Jul 17 11:47:14.686572 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.686556    2566 scope.go:117] "RemoveContainer" containerID="893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56"
Jul 17 11:47:14.686876 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:14.686856    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56\": container with ID starting with 893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56 not found: ID does not exist" containerID="893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56"
Jul 17 11:47:14.686935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.686884    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56"} err="failed to get container status \"893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56\": rpc error: code = NotFound desc = could not find container \"893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56\": container with ID starting with 893048fc5f0cffb0ae791e6ce05957c9979552a652edbfa228ab91548bf84a56 not found: ID does not exist"
Jul 17 11:47:14.686935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.686901    2566 scope.go:117] "RemoveContainer" containerID="cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07"
Jul 17 11:47:14.687143 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:14.687126    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07\": container with ID starting with cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07 not found: ID does not exist" containerID="cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07"
Jul 17 11:47:14.687181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.687149    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07"} err="failed to get container status \"cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07\": rpc error: code = NotFound desc = could not find container \"cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07\": container with ID starting with cf679e28a9a4d325ea847ffff15a7bb5693868dcb2f1842377002f6e679daf07 not found: ID does not exist"
Jul 17 11:47:14.701207 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.701185    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"]
Jul 17 11:47:14.709905 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:14.709879    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a50d5b-openshift-default-d4f4b7667-cldcj"]
Jul 17 11:47:15.657805 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:15.657720    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" path="/var/lib/kubelet/pods/6b8b6296-1cd3-4010-ad98-9156ec28e3a5/volumes"
Jul 17 11:47:15.667589 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:15.667564    2566 generic.go:356] "Generic (PLEG): container finished" podID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerID="ec3a4697d61cd4230268440829b24beaef16e6c2270254b46caaa26a22bd527c" exitCode=6
Jul 17 11:47:15.667756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:15.667629    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" event={"ID":"e20fd72a-9073-4ce4-94d0-d4c2477057f9","Type":"ContainerDied","Data":"ec3a4697d61cd4230268440829b24beaef16e6c2270254b46caaa26a22bd527c"}
Jul 17 11:47:15.667756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:15.667685    2566 scope.go:117] "RemoveContainer" containerID="308cc87e249b5a39d5d9189cbc9e3b2fd7a0268850347a822f3edc4e62cac8ad"
Jul 17 11:47:15.667984 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:15.667971    2566 scope.go:117] "RemoveContainer" containerID="ec3a4697d61cd4230268440829b24beaef16e6c2270254b46caaa26a22bd527c"
Jul 17 11:47:15.668189 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:15.668163    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-r48vg_odh-ai-gateway-infra(e20fd72a-9073-4ce4-94d0-d4c2477057f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9"
Jul 17 11:47:16.696319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:16.696284    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"]
Jul 17 11:47:16.827871 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:16.827845    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"
Jul 17 11:47:16.946873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:16.946791    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20fd72a-9073-4ce4-94d0-d4c2477057f9-maas-api-service-ca\") pod \"e20fd72a-9073-4ce4-94d0-d4c2477057f9\" (UID: \"e20fd72a-9073-4ce4-94d0-d4c2477057f9\") "
Jul 17 11:47:16.947181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:16.947162    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20fd72a-9073-4ce4-94d0-d4c2477057f9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e20fd72a-9073-4ce4-94d0-d4c2477057f9" (UID: "e20fd72a-9073-4ce4-94d0-d4c2477057f9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:47:17.047565 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:17.047533    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20fd72a-9073-4ce4-94d0-d4c2477057f9-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:17.679832 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:17.679809    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"
Jul 17 11:47:17.679963 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:17.679864    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg" event={"ID":"e20fd72a-9073-4ce4-94d0-d4c2477057f9","Type":"ContainerDied","Data":"acceffd18acc03b88db71460777224b4985b15966ef83d16659cf6815e20f3dc"}
Jul 17 11:47:17.679963 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:17.679916    2566 scope.go:117] "RemoveContainer" containerID="ec3a4697d61cd4230268440829b24beaef16e6c2270254b46caaa26a22bd527c"
Jul 17 11:47:17.699634 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:17.699612    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"]
Jul 17 11:47:17.701600 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:17.701579    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-r48vg"]
Jul 17 11:47:18.077421 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077389    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"]
Jul 17 11:47:18.077820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077806    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerName="revoke-keys"
Jul 17 11:47:18.077874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077822    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerName="revoke-keys"
Jul 17 11:47:18.077874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077830    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy"
Jul 17 11:47:18.077874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077835    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy"
Jul 17 11:47:18.077874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077849    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy"
Jul 17 11:47:18.077874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077854    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy"
Jul 17 11:47:18.077874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077861    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerName="revoke-keys"
Jul 17 11:47:18.077874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077866    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerName="revoke-keys"
Jul 17 11:47:18.078089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077884    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerName="revoke-keys"
Jul 17 11:47:18.078089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077890    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerName="revoke-keys"
Jul 17 11:47:18.078089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077961    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerName="revoke-keys"
Jul 17 11:47:18.078089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077970    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerName="revoke-keys"
Jul 17 11:47:18.078089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077974    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" containerName="revoke-keys"
Jul 17 11:47:18.078089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077979    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy"
Jul 17 11:47:18.078089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.077989    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b8b6296-1cd3-4010-ad98-9156ec28e3a5" containerName="istio-proxy"
Jul 17 11:47:18.082366 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.082342    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.085000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.084979    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-9d836f-openshift-default-dockercfg-2bnmf\""
Jul 17 11:47:18.095808 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.095781    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"]
Jul 17 11:47:18.147224 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.147191    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:47:18.259866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.259836    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-envoy\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.260052 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.259875    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e4e997-bfb1-417a-a59f-bc38972a15ad-istiod-ca-cert\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.260052 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.259968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-certs\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.260052 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.260001    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-socket\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.260052 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.260028    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-token\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.260197 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.260078    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-podinfo\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.260197 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.260152    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cktxd\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-kube-api-access-cktxd\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.260197 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.260179    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-credential-socket\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.260304 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.260227    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-data\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.360964 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.360888    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-podinfo\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.360964 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.360935    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cktxd\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-kube-api-access-cktxd\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.360964 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.360957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-credential-socket\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.361198 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361075    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-data\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.361198 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361115    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-envoy\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.361198 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e4e997-bfb1-417a-a59f-bc38972a15ad-istiod-ca-cert\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.361198 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361188    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-certs\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.361420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-socket\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.361420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-token\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.361420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361324    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-credential-socket\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.361420 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361387    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-data\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.361933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361910    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-socket\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.362081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361975    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e4e997-bfb1-417a-a59f-bc38972a15ad-istiod-ca-cert\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.362081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.361985    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-certs\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.363396 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.363376    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-podinfo\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.363620 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.363602    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-envoy\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.370019 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.370001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-token\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.370205 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.370185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cktxd\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-kube-api-access-cktxd\") pod \"e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") " pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.393084 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.393065    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:18.561646 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.561614    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"]
Jul 17 11:47:18.563009 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:47:18.562983    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e4e997_bfb1_417a_a59f_bc38972a15ad.slice/crio-addeda30d17795793f11328040d1ef417942f4d587e576cdda556c9305d13ed0 WatchSource:0}: Error finding container addeda30d17795793f11328040d1ef417942f4d587e576cdda556c9305d13ed0: Status 404 returned error can't find the container with id addeda30d17795793f11328040d1ef417942f4d587e576cdda556c9305d13ed0
Jul 17 11:47:18.565563 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.565529    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:47:18.565662 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.565599    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:47:18.565662 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.565629    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:47:18.688258 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.688221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" event={"ID":"36e4e997-bfb1-417a-a59f-bc38972a15ad","Type":"ContainerStarted","Data":"f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd"}
Jul 17 11:47:18.688502 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.688272    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" event={"ID":"36e4e997-bfb1-417a-a59f-bc38972a15ad","Type":"ContainerStarted","Data":"addeda30d17795793f11328040d1ef417942f4d587e576cdda556c9305d13ed0"}
Jul 17 11:47:18.720270 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:18.720168    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" podStartSLOduration=0.720146662 podStartE2EDuration="720.146662ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:47:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:47:18.716650558 +0000 UTC m=+3021.573734503" watchObservedRunningTime="2026-07-17 11:47:18.720146662 +0000 UTC m=+3021.577230606"
Jul 17 11:47:19.113865 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.113828    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"]
Jul 17 11:47:19.117446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.117428    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"
Jul 17 11:47:19.129723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.129676    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"]
Jul 17 11:47:19.270117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.270083    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-d8d49\" (UID: \"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"
Jul 17 11:47:19.371615 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.371535    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-d8d49\" (UID: \"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"
Jul 17 11:47:19.372163 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.372144    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-d8d49\" (UID: \"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"
Jul 17 11:47:19.393616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.393587    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:19.398656 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.398633    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:19.427744 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.427701    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"
Jul 17 11:47:19.657574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.657500    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e20fd72a-9073-4ce4-94d0-d4c2477057f9" path="/var/lib/kubelet/pods/e20fd72a-9073-4ce4-94d0-d4c2477057f9/volumes"
Jul 17 11:47:19.693856 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.693828    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:19.695252 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.695230    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:19.777660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:19.777636    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"]
Jul 17 11:47:19.778258 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:47:19.778232    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe7ca55_5ea4_4f7e_9805_d1a38a6d4cb2.slice/crio-961d69b8899f04f5d4a4b8101bfe0657f809832f742939df1f1bb226f316d6f3 WatchSource:0}: Error finding container 961d69b8899f04f5d4a4b8101bfe0657f809832f742939df1f1bb226f316d6f3: Status 404 returned error can't find the container with id 961d69b8899f04f5d4a4b8101bfe0657f809832f742939df1f1bb226f316d6f3
Jul 17 11:47:20.701326 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:20.701241    2566 generic.go:356] "Generic (PLEG): container finished" podID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerID="6aaa518fe6b58b9609bd2cb528ea2c9568455cc1ab3404d379579086bb75993b" exitCode=6
Jul 17 11:47:20.701469 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:20.701328    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49" event={"ID":"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2","Type":"ContainerDied","Data":"6aaa518fe6b58b9609bd2cb528ea2c9568455cc1ab3404d379579086bb75993b"}
Jul 17 11:47:20.701469 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:20.701374    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49" event={"ID":"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2","Type":"ContainerStarted","Data":"961d69b8899f04f5d4a4b8101bfe0657f809832f742939df1f1bb226f316d6f3"}
Jul 17 11:47:20.701543 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:20.701526    2566 scope.go:117] "RemoveContainer" containerID="6aaa518fe6b58b9609bd2cb528ea2c9568455cc1ab3404d379579086bb75993b"
Jul 17 11:47:21.708625 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:21.708538    2566 generic.go:356] "Generic (PLEG): container finished" podID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerID="38874282ebb0ff0c156449657ac9ea2f45302302c128d8ad799039e96a9634f3" exitCode=6
Jul 17 11:47:21.709122 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:21.708621    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49" event={"ID":"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2","Type":"ContainerDied","Data":"38874282ebb0ff0c156449657ac9ea2f45302302c128d8ad799039e96a9634f3"}
Jul 17 11:47:21.709122 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:21.708680    2566 scope.go:117] "RemoveContainer" containerID="6aaa518fe6b58b9609bd2cb528ea2c9568455cc1ab3404d379579086bb75993b"
Jul 17 11:47:21.709122 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:21.708943    2566 scope.go:117] "RemoveContainer" containerID="38874282ebb0ff0c156449657ac9ea2f45302302c128d8ad799039e96a9634f3"
Jul 17 11:47:21.709306 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:21.709167    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-d8d49_odh-ai-gateway-infra(abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2"
Jul 17 11:47:22.715975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:22.715944    2566 scope.go:117] "RemoveContainer" containerID="38874282ebb0ff0c156449657ac9ea2f45302302c128d8ad799039e96a9634f3"
Jul 17 11:47:22.716357 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:22.716188    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-d8d49_odh-ai-gateway-infra(abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2"
Jul 17 11:47:23.652552 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:23.652519    2566 scope.go:117] "RemoveContainer" containerID="7750fbe1a0aee533508ea7b0cfb0f16c92d23e7aa28da0790f0242b680419189"
Jul 17 11:47:24.728060 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:24.727982    2566 generic.go:356] "Generic (PLEG): container finished" podID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerID="df6f4038fdca252386b91418fd90f71672bbb6ea00383249d80c1cbf7dddb339" exitCode=6
Jul 17 11:47:24.728060 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:24.728052    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj" event={"ID":"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7","Type":"ContainerDied","Data":"df6f4038fdca252386b91418fd90f71672bbb6ea00383249d80c1cbf7dddb339"}
Jul 17 11:47:24.728497 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:24.728109    2566 scope.go:117] "RemoveContainer" containerID="7750fbe1a0aee533508ea7b0cfb0f16c92d23e7aa28da0790f0242b680419189"
Jul 17 11:47:24.728497 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:24.728360    2566 scope.go:117] "RemoveContainer" containerID="df6f4038fdca252386b91418fd90f71672bbb6ea00383249d80c1cbf7dddb339"
Jul 17 11:47:24.728653 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:24.728628    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-dk4xj_odh-ai-gateway-infra(5421a860-d1cb-4fc1-b2d2-29b4fa6974f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7"
Jul 17 11:47:25.760513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:25.760473    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"]
Jul 17 11:47:25.889341 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:25.889313    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"
Jul 17 11:47:25.940120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:25.940082    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5421a860-d1cb-4fc1-b2d2-29b4fa6974f7-maas-api-service-ca\") pod \"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7\" (UID: \"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7\") "
Jul 17 11:47:25.940391 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:25.940370    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5421a860-d1cb-4fc1-b2d2-29b4fa6974f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" (UID: "5421a860-d1cb-4fc1-b2d2-29b4fa6974f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:47:26.041029 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:26.040955    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5421a860-d1cb-4fc1-b2d2-29b4fa6974f7-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:26.740434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:26.740410    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"
Jul 17 11:47:26.740434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:26.740431    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj" event={"ID":"5421a860-d1cb-4fc1-b2d2-29b4fa6974f7","Type":"ContainerDied","Data":"95760b416bbfceb34825d61438f3e85fad1a96c10042cda4dee87d38212fac79"}
Jul 17 11:47:26.740666 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:26.740483    2566 scope.go:117] "RemoveContainer" containerID="df6f4038fdca252386b91418fd90f71672bbb6ea00383249d80c1cbf7dddb339"
Jul 17 11:47:26.768906 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:26.768885    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"]
Jul 17 11:47:26.777075 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:26.777056    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-dk4xj"]
Jul 17 11:47:27.657397 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:27.657366    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" path="/var/lib/kubelet/pods/5421a860-d1cb-4fc1-b2d2-29b4fa6974f7/volumes"
Jul 17 11:47:30.853616 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:30.853580    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:47:34.652636 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:34.652601    2566 scope.go:117] "RemoveContainer" containerID="38874282ebb0ff0c156449657ac9ea2f45302302c128d8ad799039e96a9634f3"
Jul 17 11:47:35.789285 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:35.789255    2566 generic.go:356] "Generic (PLEG): container finished" podID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerID="6b8d2fb09d198bdcb45234d4d1c6cea03203ac36657f3ea28ca03b0d8c82854e" exitCode=6
Jul 17 11:47:35.789721 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:35.789324    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49" event={"ID":"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2","Type":"ContainerDied","Data":"6b8d2fb09d198bdcb45234d4d1c6cea03203ac36657f3ea28ca03b0d8c82854e"}
Jul 17 11:47:35.789721 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:35.789388    2566 scope.go:117] "RemoveContainer" containerID="38874282ebb0ff0c156449657ac9ea2f45302302c128d8ad799039e96a9634f3"
Jul 17 11:47:35.789868 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:35.789810    2566 scope.go:117] "RemoveContainer" containerID="6b8d2fb09d198bdcb45234d4d1c6cea03203ac36657f3ea28ca03b0d8c82854e"
Jul 17 11:47:35.790057 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:35.790037    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-d8d49_odh-ai-gateway-infra(abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2"
Jul 17 11:47:36.825742 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:36.825696    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"]
Jul 17 11:47:36.981673 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:36.981649    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"
Jul 17 11:47:37.044339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:37.044297    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2-maas-api-service-ca\") pod \"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2\" (UID: \"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2\") "
Jul 17 11:47:37.044595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:37.044575    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" (UID: "abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:47:37.044750 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:37.044735    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:47:37.803698 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:37.803666    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49" event={"ID":"abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2","Type":"ContainerDied","Data":"961d69b8899f04f5d4a4b8101bfe0657f809832f742939df1f1bb226f316d6f3"}
Jul 17 11:47:37.803879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:37.803745    2566 scope.go:117] "RemoveContainer" containerID="6b8d2fb09d198bdcb45234d4d1c6cea03203ac36657f3ea28ca03b0d8c82854e"
Jul 17 11:47:37.803879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:37.803679    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"
Jul 17 11:47:37.823437 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:37.823415    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"]
Jul 17 11:47:37.827934 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:37.827906    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-d8d49"]
Jul 17 11:47:38.810723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:38.810631    2566 generic.go:356] "Generic (PLEG): container finished" podID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerID="f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd" exitCode=0
Jul 17 11:47:38.810898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:38.810728    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" event={"ID":"36e4e997-bfb1-417a-a59f-bc38972a15ad","Type":"ContainerDied","Data":"f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd"}
Jul 17 11:47:38.811148 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:38.811125    2566 scope.go:117] "RemoveContainer" containerID="f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd"
Jul 17 11:47:38.812134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:38.812110    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:47:38.812199 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:38.812169    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:47:38.812235 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:38.812199    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 11:47:39.662416 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:39.662377    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" path="/var/lib/kubelet/pods/abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2/volumes"
Jul 17 11:47:39.819726 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:39.819677    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" event={"ID":"36e4e997-bfb1-417a-a59f-bc38972a15ad","Type":"ContainerStarted","Data":"dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366"}
Jul 17 11:47:40.394029 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:40.393995    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:40.394209 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:40.394190    2566 patch_prober.go:28] interesting pod/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Jul 17 11:47:40.394254 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:40.394237    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Jul 17 11:47:41.394032 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:41.393992    2566 patch_prober.go:28] interesting pod/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Jul 17 11:47:41.394520 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:41.394067    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Jul 17 11:47:42.394279 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:42.394241    2566 patch_prober.go:28] interesting pod/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Jul 17 11:47:42.394668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:42.394299    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Jul 17 11:47:43.393562 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:43.393530    2566 patch_prober.go:28] interesting pod/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Jul 17 11:47:43.393771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:43.393582    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Jul 17 11:47:44.397407 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:44.397376    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:44.397837 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:44.397735    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:44.398434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:44.398416    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:47:48.683107 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:48.683073    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:47:49.025677 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.025595    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"]
Jul 17 11:47:49.026052 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026035    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerName="revoke-keys"
Jul 17 11:47:49.026113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026054    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerName="revoke-keys"
Jul 17 11:47:49.026113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026062    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerName="revoke-keys"
Jul 17 11:47:49.026113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026068    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerName="revoke-keys"
Jul 17 11:47:49.026113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026090    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerName="revoke-keys"
Jul 17 11:47:49.026113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026096    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerName="revoke-keys"
Jul 17 11:47:49.026113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026110    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerName="revoke-keys"
Jul 17 11:47:49.026113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026115    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerName="revoke-keys"
Jul 17 11:47:49.026358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026169    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerName="revoke-keys"
Jul 17 11:47:49.026358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026177    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerName="revoke-keys"
Jul 17 11:47:49.026358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026183    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerName="revoke-keys"
Jul 17 11:47:49.026358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026191    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerName="revoke-keys"
Jul 17 11:47:49.026358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.026197    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerName="revoke-keys"
Jul 17 11:47:49.029594 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.029572    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"
Jul 17 11:47:49.032767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.032749    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:47:49.032862 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.032848    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:47:49.039334 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.039308    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"]
Jul 17 11:47:49.155663 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.155635    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e166a5-2884-4d9d-8deb-67d23f609621-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl\" (UID: \"53e166a5-2884-4d9d-8deb-67d23f609621\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"
Jul 17 11:47:49.256896 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.256869    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e166a5-2884-4d9d-8deb-67d23f609621-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl\" (UID: \"53e166a5-2884-4d9d-8deb-67d23f609621\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"
Jul 17 11:47:49.257438 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.257422    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e166a5-2884-4d9d-8deb-67d23f609621-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl\" (UID: \"53e166a5-2884-4d9d-8deb-67d23f609621\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"
Jul 17 11:47:49.341386 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.341360    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"
Jul 17 11:47:49.486205 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.486172    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"]
Jul 17 11:47:49.488879 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:47:49.488849    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e166a5_2884_4d9d_8deb_67d23f609621.slice/crio-608178e38cacd0bdd1d3cc2a64ca32ef1c650d98a97ddbb42316314cf7ed6712 WatchSource:0}: Error finding container 608178e38cacd0bdd1d3cc2a64ca32ef1c650d98a97ddbb42316314cf7ed6712: Status 404 returned error can't find the container with id 608178e38cacd0bdd1d3cc2a64ca32ef1c650d98a97ddbb42316314cf7ed6712
Jul 17 11:47:49.878088 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.878058    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" event={"ID":"53e166a5-2884-4d9d-8deb-67d23f609621","Type":"ContainerStarted","Data":"e38eb50cceb79b9670950c7290be0e400cd098ae17f955ee409eec0646ee72e4"}
Jul 17 11:47:49.878401 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.878117    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" event={"ID":"53e166a5-2884-4d9d-8deb-67d23f609621","Type":"ContainerStarted","Data":"608178e38cacd0bdd1d3cc2a64ca32ef1c650d98a97ddbb42316314cf7ed6712"}
Jul 17 11:47:49.914208 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:49.914165    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" podStartSLOduration=0.9141493 podStartE2EDuration="914.1493ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:47:49.91142452 +0000 UTC m=+3052.768508486" watchObservedRunningTime="2026-07-17 11:47:49.9141493 +0000 UTC m=+3052.771233245"
Jul 17 11:47:50.885378 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:50.885345    2566 generic.go:356] "Generic (PLEG): container finished" podID="53e166a5-2884-4d9d-8deb-67d23f609621" containerID="e38eb50cceb79b9670950c7290be0e400cd098ae17f955ee409eec0646ee72e4" exitCode=6
Jul 17 11:47:50.885794 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:50.885432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" event={"ID":"53e166a5-2884-4d9d-8deb-67d23f609621","Type":"ContainerDied","Data":"e38eb50cceb79b9670950c7290be0e400cd098ae17f955ee409eec0646ee72e4"}
Jul 17 11:47:50.885794 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:50.885593    2566 scope.go:117] "RemoveContainer" containerID="e38eb50cceb79b9670950c7290be0e400cd098ae17f955ee409eec0646ee72e4"
Jul 17 11:47:51.892859 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:51.892819    2566 generic.go:356] "Generic (PLEG): container finished" podID="53e166a5-2884-4d9d-8deb-67d23f609621" containerID="c163db28c172b4c00f110e0d94e075e62d9ebaf6b5b64d64b2f66b0a6f9bd5b3" exitCode=6
Jul 17 11:47:51.893283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:51.892903    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" event={"ID":"53e166a5-2884-4d9d-8deb-67d23f609621","Type":"ContainerDied","Data":"c163db28c172b4c00f110e0d94e075e62d9ebaf6b5b64d64b2f66b0a6f9bd5b3"}
Jul 17 11:47:51.893283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:51.892958    2566 scope.go:117] "RemoveContainer" containerID="e38eb50cceb79b9670950c7290be0e400cd098ae17f955ee409eec0646ee72e4"
Jul 17 11:47:51.893283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:51.893178    2566 scope.go:117] "RemoveContainer" containerID="c163db28c172b4c00f110e0d94e075e62d9ebaf6b5b64d64b2f66b0a6f9bd5b3"
Jul 17 11:47:51.893428 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:51.893392    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl_odh-ai-gateway-infra(53e166a5-2884-4d9d-8deb-67d23f609621)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" podUID="53e166a5-2884-4d9d-8deb-67d23f609621"
Jul 17 11:47:52.900359 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:52.900333    2566 scope.go:117] "RemoveContainer" containerID="c163db28c172b4c00f110e0d94e075e62d9ebaf6b5b64d64b2f66b0a6f9bd5b3"
Jul 17 11:47:52.900746 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:52.900513    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl_odh-ai-gateway-infra(53e166a5-2884-4d9d-8deb-67d23f609621)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" podUID="53e166a5-2884-4d9d-8deb-67d23f609621"
Jul 17 11:47:55.855522 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:55.855491    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"]
Jul 17 11:47:55.855942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:55.855884    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerName="revoke-keys"
Jul 17 11:47:55.855942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:55.855896    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerName="revoke-keys"
Jul 17 11:47:55.855942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:55.855905    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerName="revoke-keys"
Jul 17 11:47:55.855942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:55.855910    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="abe7ca55-5ea4-4f7e-9805-d1a38a6d4cb2" containerName="revoke-keys"
Jul 17 11:47:55.856079 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:55.855983    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5421a860-d1cb-4fc1-b2d2-29b4fa6974f7" containerName="revoke-keys"
Jul 17 11:47:55.859836 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:55.859821    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"
Jul 17 11:47:55.867367 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:55.867331    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"]
Jul 17 11:47:56.014362 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:56.014336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdddcc79-e79c-4a37-ada9-cdca41fb79e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-k4qgx\" (UID: \"bdddcc79-e79c-4a37-ada9-cdca41fb79e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"
Jul 17 11:47:56.115117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:56.115041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdddcc79-e79c-4a37-ada9-cdca41fb79e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-k4qgx\" (UID: \"bdddcc79-e79c-4a37-ada9-cdca41fb79e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"
Jul 17 11:47:56.115653 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:56.115635    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdddcc79-e79c-4a37-ada9-cdca41fb79e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-k4qgx\" (UID: \"bdddcc79-e79c-4a37-ada9-cdca41fb79e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"
Jul 17 11:47:56.172077 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:56.172045    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"
Jul 17 11:47:56.300275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:56.300250    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"]
Jul 17 11:47:56.302349 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:47:56.302322    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdddcc79_e79c_4a37_ada9_cdca41fb79e8.slice/crio-fc5a569ce70a4d954f82e100dc538ba4271b397e934222b1e87e8e37c9de6195 WatchSource:0}: Error finding container fc5a569ce70a4d954f82e100dc538ba4271b397e934222b1e87e8e37c9de6195: Status 404 returned error can't find the container with id fc5a569ce70a4d954f82e100dc538ba4271b397e934222b1e87e8e37c9de6195
Jul 17 11:47:56.922442 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:56.922404    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" event={"ID":"bdddcc79-e79c-4a37-ada9-cdca41fb79e8","Type":"ContainerStarted","Data":"12d520711cbcf6cf6bcadbc9f132d96eeb8f77c1ee6f5203ddadede9ee97e8c0"}
Jul 17 11:47:56.922857 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:56.922455    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" event={"ID":"bdddcc79-e79c-4a37-ada9-cdca41fb79e8","Type":"ContainerStarted","Data":"fc5a569ce70a4d954f82e100dc538ba4271b397e934222b1e87e8e37c9de6195"}
Jul 17 11:47:56.944163 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:56.944115    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" podStartSLOduration=1.9440963679999999 podStartE2EDuration="1.944096368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:47:56.942032392 +0000 UTC m=+3059.799116339" watchObservedRunningTime="2026-07-17 11:47:56.944096368 +0000 UTC m=+3059.801180313"
Jul 17 11:47:57.929641 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:57.929610    2566 generic.go:356] "Generic (PLEG): container finished" podID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerID="12d520711cbcf6cf6bcadbc9f132d96eeb8f77c1ee6f5203ddadede9ee97e8c0" exitCode=6
Jul 17 11:47:57.930061 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:57.929718    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" event={"ID":"bdddcc79-e79c-4a37-ada9-cdca41fb79e8","Type":"ContainerDied","Data":"12d520711cbcf6cf6bcadbc9f132d96eeb8f77c1ee6f5203ddadede9ee97e8c0"}
Jul 17 11:47:57.930061 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:57.929922    2566 scope.go:117] "RemoveContainer" containerID="12d520711cbcf6cf6bcadbc9f132d96eeb8f77c1ee6f5203ddadede9ee97e8c0"
Jul 17 11:47:58.044720 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.044685    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"]
Jul 17 11:47:58.049257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.049229    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"
Jul 17 11:47:58.056281 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.056178    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"]
Jul 17 11:47:58.131132 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.131092    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc377c-f2ca-447a-bd62-99d205812e86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-7x99p\" (UID: \"bcbc377c-f2ca-447a-bd62-99d205812e86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"
Jul 17 11:47:58.232024 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.231997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc377c-f2ca-447a-bd62-99d205812e86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-7x99p\" (UID: \"bcbc377c-f2ca-447a-bd62-99d205812e86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"
Jul 17 11:47:58.232577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.232555    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc377c-f2ca-447a-bd62-99d205812e86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-7x99p\" (UID: \"bcbc377c-f2ca-447a-bd62-99d205812e86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"
Jul 17 11:47:58.361506 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.361454    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"
Jul 17 11:47:58.491699 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.491663    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"]
Jul 17 11:47:58.493800 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:47:58.493760    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbc377c_f2ca_447a_bd62_99d205812e86.slice/crio-0915ae16bcc992611dad542b0f28945c93a523811beb548fdec64f83944ad5c5 WatchSource:0}: Error finding container 0915ae16bcc992611dad542b0f28945c93a523811beb548fdec64f83944ad5c5: Status 404 returned error can't find the container with id 0915ae16bcc992611dad542b0f28945c93a523811beb548fdec64f83944ad5c5
Jul 17 11:47:58.937796 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.937761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" event={"ID":"bcbc377c-f2ca-447a-bd62-99d205812e86","Type":"ContainerStarted","Data":"f4b8b470b8c00ceedcdd669b8bb24143c04f379165f1d6139c03ee28884ea149"}
Jul 17 11:47:58.938238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.937820    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" event={"ID":"bcbc377c-f2ca-447a-bd62-99d205812e86","Type":"ContainerStarted","Data":"0915ae16bcc992611dad542b0f28945c93a523811beb548fdec64f83944ad5c5"}
Jul 17 11:47:58.939577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.939549    2566 generic.go:356] "Generic (PLEG): container finished" podID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerID="9198bb708c52505b52e1e30cf673f7f9cdde129b2a2c48f34a0b02fba3433dc8" exitCode=6
Jul 17 11:47:58.939700 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.939594    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" event={"ID":"bdddcc79-e79c-4a37-ada9-cdca41fb79e8","Type":"ContainerDied","Data":"9198bb708c52505b52e1e30cf673f7f9cdde129b2a2c48f34a0b02fba3433dc8"}
Jul 17 11:47:58.939700 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.939638    2566 scope.go:117] "RemoveContainer" containerID="12d520711cbcf6cf6bcadbc9f132d96eeb8f77c1ee6f5203ddadede9ee97e8c0"
Jul 17 11:47:58.939836 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.939815    2566 scope.go:117] "RemoveContainer" containerID="9198bb708c52505b52e1e30cf673f7f9cdde129b2a2c48f34a0b02fba3433dc8"
Jul 17 11:47:58.940016 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:58.940000    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-k4qgx_odh-ai-gateway-infra(bdddcc79-e79c-4a37-ada9-cdca41fb79e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8"
Jul 17 11:47:58.960954 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:58.960914    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" podStartSLOduration=0.960900422 podStartE2EDuration="960.900422ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:47:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:47:58.959140562 +0000 UTC m=+3061.816224503" watchObservedRunningTime="2026-07-17 11:47:58.960900422 +0000 UTC m=+3061.817984368"
Jul 17 11:47:59.946477 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:59.946440    2566 generic.go:356] "Generic (PLEG): container finished" podID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerID="f4b8b470b8c00ceedcdd669b8bb24143c04f379165f1d6139c03ee28884ea149" exitCode=6
Jul 17 11:47:59.947066 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:59.946530    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" event={"ID":"bcbc377c-f2ca-447a-bd62-99d205812e86","Type":"ContainerDied","Data":"f4b8b470b8c00ceedcdd669b8bb24143c04f379165f1d6139c03ee28884ea149"}
Jul 17 11:47:59.947066 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:59.946724    2566 scope.go:117] "RemoveContainer" containerID="f4b8b470b8c00ceedcdd669b8bb24143c04f379165f1d6139c03ee28884ea149"
Jul 17 11:47:59.948467 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:47:59.948421    2566 scope.go:117] "RemoveContainer" containerID="9198bb708c52505b52e1e30cf673f7f9cdde129b2a2c48f34a0b02fba3433dc8"
Jul 17 11:47:59.948590 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:47:59.948571    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-k4qgx_odh-ai-gateway-infra(bdddcc79-e79c-4a37-ada9-cdca41fb79e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8"
Jul 17 11:48:00.955935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:00.955900    2566 generic.go:356] "Generic (PLEG): container finished" podID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerID="634e31595d22259d33600b3d412a5ed94fab232e6282b5da3b0be65e82e86cff" exitCode=6
Jul 17 11:48:00.956641 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:00.955987    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" event={"ID":"bcbc377c-f2ca-447a-bd62-99d205812e86","Type":"ContainerDied","Data":"634e31595d22259d33600b3d412a5ed94fab232e6282b5da3b0be65e82e86cff"}
Jul 17 11:48:00.956641 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:00.956056    2566 scope.go:117] "RemoveContainer" containerID="f4b8b470b8c00ceedcdd669b8bb24143c04f379165f1d6139c03ee28884ea149"
Jul 17 11:48:00.956641 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:00.956143    2566 scope.go:117] "RemoveContainer" containerID="634e31595d22259d33600b3d412a5ed94fab232e6282b5da3b0be65e82e86cff"
Jul 17 11:48:00.956641 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:00.956383    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-7x99p_odh-ai-gateway-infra(bcbc377c-f2ca-447a-bd62-99d205812e86)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86"
Jul 17 11:48:01.963438 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:01.963412    2566 scope.go:117] "RemoveContainer" containerID="634e31595d22259d33600b3d412a5ed94fab232e6282b5da3b0be65e82e86cff"
Jul 17 11:48:01.963909 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:01.963697    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-7x99p_odh-ai-gateway-infra(bcbc377c-f2ca-447a-bd62-99d205812e86)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86"
Jul 17 11:48:08.651607 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:08.651578    2566 scope.go:117] "RemoveContainer" containerID="c163db28c172b4c00f110e0d94e075e62d9ebaf6b5b64d64b2f66b0a6f9bd5b3"
Jul 17 11:48:09.000689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:09.000598    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" event={"ID":"53e166a5-2884-4d9d-8deb-67d23f609621","Type":"ContainerStarted","Data":"45f430b64f04473d9325d55b573749c7f5bc10e0998853332453c84f82387883"}
Jul 17 11:48:09.151257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:09.151226    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"]
Jul 17 11:48:09.154496 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:09.154480    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"
Jul 17 11:48:09.163143 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:09.163118    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"]
Jul 17 11:48:09.328295 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:09.328271    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed1c47a-f97c-4aae-bf61-4fc5e4b052d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj\" (UID: \"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"
Jul 17 11:48:09.429695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:09.429656    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed1c47a-f97c-4aae-bf61-4fc5e4b052d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj\" (UID: \"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"
Jul 17 11:48:09.430283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:09.430256    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed1c47a-f97c-4aae-bf61-4fc5e4b052d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj\" (UID: \"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"
Jul 17 11:48:09.466407 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:09.466380    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"
Jul 17 11:48:09.597831 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:09.597804    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"]
Jul 17 11:48:09.600155 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:48:09.600126    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed1c47a_f97c_4aae_bf61_4fc5e4b052d7.slice/crio-8a3a3857e4b980743ef71125e77753bc6e93959a1714a5e128e15880d6147b97 WatchSource:0}: Error finding container 8a3a3857e4b980743ef71125e77753bc6e93959a1714a5e128e15880d6147b97: Status 404 returned error can't find the container with id 8a3a3857e4b980743ef71125e77753bc6e93959a1714a5e128e15880d6147b97
Jul 17 11:48:10.007785 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:10.007756    2566 generic.go:356] "Generic (PLEG): container finished" podID="53e166a5-2884-4d9d-8deb-67d23f609621" containerID="45f430b64f04473d9325d55b573749c7f5bc10e0998853332453c84f82387883" exitCode=6
Jul 17 11:48:10.008184 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:10.007833    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" event={"ID":"53e166a5-2884-4d9d-8deb-67d23f609621","Type":"ContainerDied","Data":"45f430b64f04473d9325d55b573749c7f5bc10e0998853332453c84f82387883"}
Jul 17 11:48:10.008184 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:10.007891    2566 scope.go:117] "RemoveContainer" containerID="c163db28c172b4c00f110e0d94e075e62d9ebaf6b5b64d64b2f66b0a6f9bd5b3"
Jul 17 11:48:10.008286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:10.008191    2566 scope.go:117] "RemoveContainer" containerID="45f430b64f04473d9325d55b573749c7f5bc10e0998853332453c84f82387883"
Jul 17 11:48:10.008428 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:10.008404    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl_odh-ai-gateway-infra(53e166a5-2884-4d9d-8deb-67d23f609621)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" podUID="53e166a5-2884-4d9d-8deb-67d23f609621"
Jul 17 11:48:10.009490 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:10.009464    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" event={"ID":"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7","Type":"ContainerStarted","Data":"b2e616c4705d449a4e206d1ad2b955e0f8fb70b2fd3209b56ae38b3afec35e55"}
Jul 17 11:48:10.009581 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:10.009516    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" event={"ID":"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7","Type":"ContainerStarted","Data":"8a3a3857e4b980743ef71125e77753bc6e93959a1714a5e128e15880d6147b97"}
Jul 17 11:48:10.027974 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:10.027953    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"]
Jul 17 11:48:10.059329 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:10.059291    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" podStartSLOduration=1.059279152 podStartE2EDuration="1.059279152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:48:10.058016229 +0000 UTC m=+3072.915100174" watchObservedRunningTime="2026-07-17 11:48:10.059279152 +0000 UTC m=+3072.916363097"
Jul 17 11:48:10.151095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:10.151067    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:48:11.016539 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:11.016507    2566 generic.go:356] "Generic (PLEG): container finished" podID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerID="b2e616c4705d449a4e206d1ad2b955e0f8fb70b2fd3209b56ae38b3afec35e55" exitCode=6
Jul 17 11:48:11.016979 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:11.016564    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" event={"ID":"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7","Type":"ContainerDied","Data":"b2e616c4705d449a4e206d1ad2b955e0f8fb70b2fd3209b56ae38b3afec35e55"}
Jul 17 11:48:11.016979 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:11.016752    2566 scope.go:117] "RemoveContainer" containerID="b2e616c4705d449a4e206d1ad2b955e0f8fb70b2fd3209b56ae38b3afec35e55"
Jul 17 11:48:11.165093 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:11.165068    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"
Jul 17 11:48:11.347942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:11.347914    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e166a5-2884-4d9d-8deb-67d23f609621-maas-api-service-ca\") pod \"53e166a5-2884-4d9d-8deb-67d23f609621\" (UID: \"53e166a5-2884-4d9d-8deb-67d23f609621\") "
Jul 17 11:48:11.348250 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:11.348230    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53e166a5-2884-4d9d-8deb-67d23f609621-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53e166a5-2884-4d9d-8deb-67d23f609621" (UID: "53e166a5-2884-4d9d-8deb-67d23f609621"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:48:11.448695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:11.448660    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e166a5-2884-4d9d-8deb-67d23f609621-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:48:11.651535 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:11.651510    2566 scope.go:117] "RemoveContainer" containerID="9198bb708c52505b52e1e30cf673f7f9cdde129b2a2c48f34a0b02fba3433dc8"
Jul 17 11:48:12.025077 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.024999    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"
Jul 17 11:48:12.025527 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.024995    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl" event={"ID":"53e166a5-2884-4d9d-8deb-67d23f609621","Type":"ContainerDied","Data":"608178e38cacd0bdd1d3cc2a64ca32ef1c650d98a97ddbb42316314cf7ed6712"}
Jul 17 11:48:12.025527 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.025149    2566 scope.go:117] "RemoveContainer" containerID="45f430b64f04473d9325d55b573749c7f5bc10e0998853332453c84f82387883"
Jul 17 11:48:12.027010 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.026986    2566 generic.go:356] "Generic (PLEG): container finished" podID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerID="6ce4afa0fa9c57fbc87dbad03b152ea517ecdadfff59d321f554bfbe6cfb4d97" exitCode=6
Jul 17 11:48:12.027134 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.027120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" event={"ID":"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7","Type":"ContainerDied","Data":"6ce4afa0fa9c57fbc87dbad03b152ea517ecdadfff59d321f554bfbe6cfb4d97"}
Jul 17 11:48:12.027675 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.027563    2566 scope.go:117] "RemoveContainer" containerID="6ce4afa0fa9c57fbc87dbad03b152ea517ecdadfff59d321f554bfbe6cfb4d97"
Jul 17 11:48:12.027817 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:12.027796    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj_odh-ai-gateway-infra(fed1c47a-f97c-4aae-bf61-4fc5e4b052d7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7"
Jul 17 11:48:12.029594 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.029556    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" event={"ID":"bdddcc79-e79c-4a37-ada9-cdca41fb79e8","Type":"ContainerStarted","Data":"90b18276dde5d4130abc8aecbaca93672afb6a5b5c89b34ecb54a80166df5099"}
Jul 17 11:48:12.034443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.034428    2566 scope.go:117] "RemoveContainer" containerID="b2e616c4705d449a4e206d1ad2b955e0f8fb70b2fd3209b56ae38b3afec35e55"
Jul 17 11:48:12.044896 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.044869    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"]
Jul 17 11:48:12.047965 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:12.047943    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-2fnnl"]
Jul 17 11:48:13.036996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:13.036967    2566 scope.go:117] "RemoveContainer" containerID="6ce4afa0fa9c57fbc87dbad03b152ea517ecdadfff59d321f554bfbe6cfb4d97"
Jul 17 11:48:13.037455 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:13.037211    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj_odh-ai-gateway-infra(fed1c47a-f97c-4aae-bf61-4fc5e4b052d7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7"
Jul 17 11:48:13.038592 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:13.038571    2566 generic.go:356] "Generic (PLEG): container finished" podID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerID="90b18276dde5d4130abc8aecbaca93672afb6a5b5c89b34ecb54a80166df5099" exitCode=6
Jul 17 11:48:13.038736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:13.038638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" event={"ID":"bdddcc79-e79c-4a37-ada9-cdca41fb79e8","Type":"ContainerDied","Data":"90b18276dde5d4130abc8aecbaca93672afb6a5b5c89b34ecb54a80166df5099"}
Jul 17 11:48:13.038736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:13.038684    2566 scope.go:117] "RemoveContainer" containerID="9198bb708c52505b52e1e30cf673f7f9cdde129b2a2c48f34a0b02fba3433dc8"
Jul 17 11:48:13.038965 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:13.038948    2566 scope.go:117] "RemoveContainer" containerID="90b18276dde5d4130abc8aecbaca93672afb6a5b5c89b34ecb54a80166df5099"
Jul 17 11:48:13.039201 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:13.039176    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-k4qgx_odh-ai-gateway-infra(bdddcc79-e79c-4a37-ada9-cdca41fb79e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8"
Jul 17 11:48:13.071596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:13.071563    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"]
Jul 17 11:48:13.651797 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:13.651768    2566 scope.go:117] "RemoveContainer" containerID="634e31595d22259d33600b3d412a5ed94fab232e6282b5da3b0be65e82e86cff"
Jul 17 11:48:13.658020 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:13.657897    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" path="/var/lib/kubelet/pods/53e166a5-2884-4d9d-8deb-67d23f609621/volumes"
Jul 17 11:48:14.048723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:14.048677    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" event={"ID":"bcbc377c-f2ca-447a-bd62-99d205812e86","Type":"ContainerStarted","Data":"a323408da96ac5612ce69078d85b9506f3593914d8e651220b0a80077256955b"}
Jul 17 11:48:14.180678 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:14.180649    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"
Jul 17 11:48:14.272210 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:14.272176    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdddcc79-e79c-4a37-ada9-cdca41fb79e8-maas-api-service-ca\") pod \"bdddcc79-e79c-4a37-ada9-cdca41fb79e8\" (UID: \"bdddcc79-e79c-4a37-ada9-cdca41fb79e8\") "
Jul 17 11:48:14.272500 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:14.272478    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdddcc79-e79c-4a37-ada9-cdca41fb79e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bdddcc79-e79c-4a37-ada9-cdca41fb79e8" (UID: "bdddcc79-e79c-4a37-ada9-cdca41fb79e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:48:14.372924 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:14.372887    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdddcc79-e79c-4a37-ada9-cdca41fb79e8-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:48:15.056115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.056084    2566 generic.go:356] "Generic (PLEG): container finished" podID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerID="a323408da96ac5612ce69078d85b9506f3593914d8e651220b0a80077256955b" exitCode=6
Jul 17 11:48:15.056546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.056158    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" event={"ID":"bcbc377c-f2ca-447a-bd62-99d205812e86","Type":"ContainerDied","Data":"a323408da96ac5612ce69078d85b9506f3593914d8e651220b0a80077256955b"}
Jul 17 11:48:15.056546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.056217    2566 scope.go:117] "RemoveContainer" containerID="634e31595d22259d33600b3d412a5ed94fab232e6282b5da3b0be65e82e86cff"
Jul 17 11:48:15.056546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.056476    2566 scope.go:117] "RemoveContainer" containerID="a323408da96ac5612ce69078d85b9506f3593914d8e651220b0a80077256955b"
Jul 17 11:48:15.056762 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:15.056739    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-7x99p_odh-ai-gateway-infra(bcbc377c-f2ca-447a-bd62-99d205812e86)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86"
Jul 17 11:48:15.057470 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.057449    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx" event={"ID":"bdddcc79-e79c-4a37-ada9-cdca41fb79e8","Type":"ContainerDied","Data":"fc5a569ce70a4d954f82e100dc538ba4271b397e934222b1e87e8e37c9de6195"}
Jul 17 11:48:15.057548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.057480    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"
Jul 17 11:48:15.068539 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.068522    2566 scope.go:117] "RemoveContainer" containerID="90b18276dde5d4130abc8aecbaca93672afb6a5b5c89b34ecb54a80166df5099"
Jul 17 11:48:15.089933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.089906    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"]
Jul 17 11:48:15.094637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.094612    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"]
Jul 17 11:48:15.097402 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.097382    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-k4qgx"]
Jul 17 11:48:15.657410 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:15.657376    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" path="/var/lib/kubelet/pods/bdddcc79-e79c-4a37-ada9-cdca41fb79e8/volumes"
Jul 17 11:48:16.204527 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:16.204501    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"
Jul 17 11:48:16.290696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:16.290651    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc377c-f2ca-447a-bd62-99d205812e86-maas-api-service-ca\") pod \"bcbc377c-f2ca-447a-bd62-99d205812e86\" (UID: \"bcbc377c-f2ca-447a-bd62-99d205812e86\") "
Jul 17 11:48:16.291005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:16.290984    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcbc377c-f2ca-447a-bd62-99d205812e86-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bcbc377c-f2ca-447a-bd62-99d205812e86" (UID: "bcbc377c-f2ca-447a-bd62-99d205812e86"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:48:16.391305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:16.391273    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc377c-f2ca-447a-bd62-99d205812e86-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:48:17.070766 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:17.070733    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"
Jul 17 11:48:17.070766 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:17.070738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p" event={"ID":"bcbc377c-f2ca-447a-bd62-99d205812e86","Type":"ContainerDied","Data":"0915ae16bcc992611dad542b0f28945c93a523811beb548fdec64f83944ad5c5"}
Jul 17 11:48:17.071028 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:17.070815    2566 scope.go:117] "RemoveContainer" containerID="a323408da96ac5612ce69078d85b9506f3593914d8e651220b0a80077256955b"
Jul 17 11:48:17.095194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:17.095160    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"]
Jul 17 11:48:17.100770 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:17.100742    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-7x99p"]
Jul 17 11:48:17.657576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:17.657547    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" path="/var/lib/kubelet/pods/bcbc377c-f2ca-447a-bd62-99d205812e86/volumes"
Jul 17 11:48:25.245829 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:25.245792    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:48:26.651847 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:26.651813    2566 scope.go:117] "RemoveContainer" containerID="6ce4afa0fa9c57fbc87dbad03b152ea517ecdadfff59d321f554bfbe6cfb4d97"
Jul 17 11:48:27.125264 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:27.125219    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" event={"ID":"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7","Type":"ContainerStarted","Data":"97181c7a9291fed374cdfa067e918f750f9983dee37721733945ca1d43a14a57"}
Jul 17 11:48:28.133021 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:28.132991    2566 generic.go:356] "Generic (PLEG): container finished" podID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerID="97181c7a9291fed374cdfa067e918f750f9983dee37721733945ca1d43a14a57" exitCode=6
Jul 17 11:48:28.133493 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:28.133064    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" event={"ID":"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7","Type":"ContainerDied","Data":"97181c7a9291fed374cdfa067e918f750f9983dee37721733945ca1d43a14a57"}
Jul 17 11:48:28.133493 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:28.133122    2566 scope.go:117] "RemoveContainer" containerID="6ce4afa0fa9c57fbc87dbad03b152ea517ecdadfff59d321f554bfbe6cfb4d97"
Jul 17 11:48:28.133493 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:28.133428    2566 scope.go:117] "RemoveContainer" containerID="97181c7a9291fed374cdfa067e918f750f9983dee37721733945ca1d43a14a57"
Jul 17 11:48:28.133740 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:28.133687    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj_odh-ai-gateway-infra(fed1c47a-f97c-4aae-bf61-4fc5e4b052d7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7"
Jul 17 11:48:28.156968 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:28.156939    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"]
Jul 17 11:48:29.275449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:29.275426    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"
Jul 17 11:48:29.413874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:29.413787    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed1c47a-f97c-4aae-bf61-4fc5e4b052d7-maas-api-service-ca\") pod \"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7\" (UID: \"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7\") "
Jul 17 11:48:29.414129 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:29.414109    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fed1c47a-f97c-4aae-bf61-4fc5e4b052d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" (UID: "fed1c47a-f97c-4aae-bf61-4fc5e4b052d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:48:29.515215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:29.515179    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed1c47a-f97c-4aae-bf61-4fc5e4b052d7-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:48:30.147159 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:30.147123    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj" event={"ID":"fed1c47a-f97c-4aae-bf61-4fc5e4b052d7","Type":"ContainerDied","Data":"8a3a3857e4b980743ef71125e77753bc6e93959a1714a5e128e15880d6147b97"}
Jul 17 11:48:30.147366 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:30.147181    2566 scope.go:117] "RemoveContainer" containerID="97181c7a9291fed374cdfa067e918f750f9983dee37721733945ca1d43a14a57"
Jul 17 11:48:30.147366 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:30.147138    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"
Jul 17 11:48:30.166550 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:30.166527    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"]
Jul 17 11:48:30.172910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:30.172884    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-mhhpj"]
Jul 17 11:48:31.657971 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:31.657941    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" path="/var/lib/kubelet/pods/fed1c47a-f97c-4aae-bf61-4fc5e4b052d7/volumes"
Jul 17 11:48:43.649934 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:43.649895    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:48:45.908595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908561    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"]
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908930    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" containerName="revoke-keys"
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908943    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" containerName="revoke-keys"
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908951    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerName="revoke-keys"
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908957    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerName="revoke-keys"
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908965    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerName="revoke-keys"
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908971    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerName="revoke-keys"
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908981    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerName="revoke-keys"
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908987    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerName="revoke-keys"
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.908996    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerName="revoke-keys"
Jul 17 11:48:45.908998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909001    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909011    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909016    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909025    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909030    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909038    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909043    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909049    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909054    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909059    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909064    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909122    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909131    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909140    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909148    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909159    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909168    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909176    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909184    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdddcc79-e79c-4a37-ada9-cdca41fb79e8" containerName="revoke-keys"
Jul 17 11:48:45.909317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.909194    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerName="revoke-keys"
Jul 17 11:48:45.912328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.912311    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"
Jul 17 11:48:45.915412 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.915380    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:48:45.915563 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.915431    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:48:45.922069 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:45.922042    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"]
Jul 17 11:48:46.066271 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:46.066233    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099f66a9-d185-4e34-acbb-f8b2b34770db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-8cntb\" (UID: \"099f66a9-d185-4e34-acbb-f8b2b34770db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"
Jul 17 11:48:46.167183 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:46.167103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099f66a9-d185-4e34-acbb-f8b2b34770db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-8cntb\" (UID: \"099f66a9-d185-4e34-acbb-f8b2b34770db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"
Jul 17 11:48:46.167802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:46.167784    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099f66a9-d185-4e34-acbb-f8b2b34770db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-8cntb\" (UID: \"099f66a9-d185-4e34-acbb-f8b2b34770db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"
Jul 17 11:48:46.223640 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:46.223613    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"
Jul 17 11:48:46.361316 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:46.361291    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"]
Jul 17 11:48:46.363809 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:48:46.363783    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099f66a9_d185_4e34_acbb_f8b2b34770db.slice/crio-e15600623c8a2dc7b2147dfdf5ab5bc51da21ff56f955a33b930e4dde6d0d78d WatchSource:0}: Error finding container e15600623c8a2dc7b2147dfdf5ab5bc51da21ff56f955a33b930e4dde6d0d78d: Status 404 returned error can't find the container with id e15600623c8a2dc7b2147dfdf5ab5bc51da21ff56f955a33b930e4dde6d0d78d
Jul 17 11:48:47.236574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:47.236490    2566 generic.go:356] "Generic (PLEG): container finished" podID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerID="cad16377d1ebf90b40461b96ff1ce0a3cae0fc908c8645eb25c8b145d48fb146" exitCode=6
Jul 17 11:48:47.236574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:47.236532    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb" event={"ID":"099f66a9-d185-4e34-acbb-f8b2b34770db","Type":"ContainerDied","Data":"cad16377d1ebf90b40461b96ff1ce0a3cae0fc908c8645eb25c8b145d48fb146"}
Jul 17 11:48:47.236574 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:47.236568    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb" event={"ID":"099f66a9-d185-4e34-acbb-f8b2b34770db","Type":"ContainerStarted","Data":"e15600623c8a2dc7b2147dfdf5ab5bc51da21ff56f955a33b930e4dde6d0d78d"}
Jul 17 11:48:47.237066 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:47.236738    2566 scope.go:117] "RemoveContainer" containerID="cad16377d1ebf90b40461b96ff1ce0a3cae0fc908c8645eb25c8b145d48fb146"
Jul 17 11:48:48.096547 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.096512    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"]
Jul 17 11:48:48.096895 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.096882    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerName="revoke-keys"
Jul 17 11:48:48.096951 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.096897    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerName="revoke-keys"
Jul 17 11:48:48.096951 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.096918    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerName="revoke-keys"
Jul 17 11:48:48.096951 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.096924    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerName="revoke-keys"
Jul 17 11:48:48.097046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.097004    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbc377c-f2ca-447a-bd62-99d205812e86" containerName="revoke-keys"
Jul 17 11:48:48.097046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.097014    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="53e166a5-2884-4d9d-8deb-67d23f609621" containerName="revoke-keys"
Jul 17 11:48:48.097046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.097021    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed1c47a-f97c-4aae-bf61-4fc5e4b052d7" containerName="revoke-keys"
Jul 17 11:48:48.100007 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.099990    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"
Jul 17 11:48:48.109672 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.109643    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"]
Jul 17 11:48:48.183989 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.183895    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/061abc49-5601-435f-9f81-aacc57d17653-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-d9458\" (UID: \"061abc49-5601-435f-9f81-aacc57d17653\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"
Jul 17 11:48:48.243410 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.243379    2566 generic.go:356] "Generic (PLEG): container finished" podID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerID="9de1125764a83499aee90508fa6c73972e952f4e4bda7c3e0bf709814fb3b2bf" exitCode=6
Jul 17 11:48:48.243918 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.243463    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb" event={"ID":"099f66a9-d185-4e34-acbb-f8b2b34770db","Type":"ContainerDied","Data":"9de1125764a83499aee90508fa6c73972e952f4e4bda7c3e0bf709814fb3b2bf"}
Jul 17 11:48:48.243918 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.243521    2566 scope.go:117] "RemoveContainer" containerID="cad16377d1ebf90b40461b96ff1ce0a3cae0fc908c8645eb25c8b145d48fb146"
Jul 17 11:48:48.243918 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.243762    2566 scope.go:117] "RemoveContainer" containerID="9de1125764a83499aee90508fa6c73972e952f4e4bda7c3e0bf709814fb3b2bf"
Jul 17 11:48:48.244078 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:48.244024    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-8cntb_odh-ai-gateway-infra(099f66a9-d185-4e34-acbb-f8b2b34770db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db"
Jul 17 11:48:48.284827 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.284789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/061abc49-5601-435f-9f81-aacc57d17653-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-d9458\" (UID: \"061abc49-5601-435f-9f81-aacc57d17653\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"
Jul 17 11:48:48.285449 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.285427    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/061abc49-5601-435f-9f81-aacc57d17653-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-d9458\" (UID: \"061abc49-5601-435f-9f81-aacc57d17653\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"
Jul 17 11:48:48.411536 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.411501    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"
Jul 17 11:48:48.544442 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:48.544413    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"]
Jul 17 11:48:48.546228 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:48:48.546200    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod061abc49_5601_435f_9f81_aacc57d17653.slice/crio-83e0d670be8a16d229e031e50840da602f073b65491a19d0f12458c48655b760 WatchSource:0}: Error finding container 83e0d670be8a16d229e031e50840da602f073b65491a19d0f12458c48655b760: Status 404 returned error can't find the container with id 83e0d670be8a16d229e031e50840da602f073b65491a19d0f12458c48655b760
Jul 17 11:48:49.065826 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.065748    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"]
Jul 17 11:48:49.069473 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.069457    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"
Jul 17 11:48:49.078191 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.078166    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"]
Jul 17 11:48:49.192562 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.192524    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47aabcee-294b-465c-8240-fba6f722ec6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm\" (UID: \"47aabcee-294b-465c-8240-fba6f722ec6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"
Jul 17 11:48:49.250458 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.250436    2566 scope.go:117] "RemoveContainer" containerID="9de1125764a83499aee90508fa6c73972e952f4e4bda7c3e0bf709814fb3b2bf"
Jul 17 11:48:49.250827 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:49.250629    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-8cntb_odh-ai-gateway-infra(099f66a9-d185-4e34-acbb-f8b2b34770db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db"
Jul 17 11:48:49.251777 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.251753    2566 generic.go:356] "Generic (PLEG): container finished" podID="061abc49-5601-435f-9f81-aacc57d17653" containerID="e7ea216e72b7c7200a0af788937ed3e220355df7d77fc7588ff51b1e6fede17f" exitCode=6
Jul 17 11:48:49.251870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.251822    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" event={"ID":"061abc49-5601-435f-9f81-aacc57d17653","Type":"ContainerDied","Data":"e7ea216e72b7c7200a0af788937ed3e220355df7d77fc7588ff51b1e6fede17f"}
Jul 17 11:48:49.251870 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.251854    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" event={"ID":"061abc49-5601-435f-9f81-aacc57d17653","Type":"ContainerStarted","Data":"83e0d670be8a16d229e031e50840da602f073b65491a19d0f12458c48655b760"}
Jul 17 11:48:49.251958 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.251945    2566 scope.go:117] "RemoveContainer" containerID="e7ea216e72b7c7200a0af788937ed3e220355df7d77fc7588ff51b1e6fede17f"
Jul 17 11:48:49.293610 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.293589    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47aabcee-294b-465c-8240-fba6f722ec6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm\" (UID: \"47aabcee-294b-465c-8240-fba6f722ec6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"
Jul 17 11:48:49.294388 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.294367    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47aabcee-294b-465c-8240-fba6f722ec6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm\" (UID: \"47aabcee-294b-465c-8240-fba6f722ec6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"
Jul 17 11:48:49.380873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.380846    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"
Jul 17 11:48:49.512607 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:49.512584    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"]
Jul 17 11:48:49.528691 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:48:49.528657    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47aabcee_294b_465c_8240_fba6f722ec6a.slice/crio-ecaf8bbd9bc116e22056cf1f2c76069470db6e9b838f48e6f7dd0c66b70f7d62 WatchSource:0}: Error finding container ecaf8bbd9bc116e22056cf1f2c76069470db6e9b838f48e6f7dd0c66b70f7d62: Status 404 returned error can't find the container with id ecaf8bbd9bc116e22056cf1f2c76069470db6e9b838f48e6f7dd0c66b70f7d62
Jul 17 11:48:50.259173 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:50.259142    2566 generic.go:356] "Generic (PLEG): container finished" podID="061abc49-5601-435f-9f81-aacc57d17653" containerID="ff22e825e5c6fa03d85563bc9ea416a4b8037222e3355251fdd27148ef61699d" exitCode=6
Jul 17 11:48:50.259577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:50.259221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" event={"ID":"061abc49-5601-435f-9f81-aacc57d17653","Type":"ContainerDied","Data":"ff22e825e5c6fa03d85563bc9ea416a4b8037222e3355251fdd27148ef61699d"}
Jul 17 11:48:50.259577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:50.259280    2566 scope.go:117] "RemoveContainer" containerID="e7ea216e72b7c7200a0af788937ed3e220355df7d77fc7588ff51b1e6fede17f"
Jul 17 11:48:50.259577 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:50.259511    2566 scope.go:117] "RemoveContainer" containerID="ff22e825e5c6fa03d85563bc9ea416a4b8037222e3355251fdd27148ef61699d"
Jul 17 11:48:50.259772 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:50.259758    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-d9458_odh-ai-gateway-infra(061abc49-5601-435f-9f81-aacc57d17653)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" podUID="061abc49-5601-435f-9f81-aacc57d17653"
Jul 17 11:48:50.260879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:50.260859    2566 generic.go:356] "Generic (PLEG): container finished" podID="47aabcee-294b-465c-8240-fba6f722ec6a" containerID="369526013e96cd1cb0394f3806d0af22f1e5e1ffc8cd2d7aa9b71775067faf60" exitCode=6
Jul 17 11:48:50.260984 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:50.260935    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm" event={"ID":"47aabcee-294b-465c-8240-fba6f722ec6a","Type":"ContainerDied","Data":"369526013e96cd1cb0394f3806d0af22f1e5e1ffc8cd2d7aa9b71775067faf60"}
Jul 17 11:48:50.260984 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:50.260977    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm" event={"ID":"47aabcee-294b-465c-8240-fba6f722ec6a","Type":"ContainerStarted","Data":"ecaf8bbd9bc116e22056cf1f2c76069470db6e9b838f48e6f7dd0c66b70f7d62"}
Jul 17 11:48:50.261140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:50.261125    2566 scope.go:117] "RemoveContainer" containerID="369526013e96cd1cb0394f3806d0af22f1e5e1ffc8cd2d7aa9b71775067faf60"
Jul 17 11:48:51.267922 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:51.267826    2566 generic.go:356] "Generic (PLEG): container finished" podID="47aabcee-294b-465c-8240-fba6f722ec6a" containerID="a38d9b49c9691d7a44dd27e92e31f2e11c86c8ec1a5bc23b37482c436e8d08a4" exitCode=6
Jul 17 11:48:51.267922 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:51.267895    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm" event={"ID":"47aabcee-294b-465c-8240-fba6f722ec6a","Type":"ContainerDied","Data":"a38d9b49c9691d7a44dd27e92e31f2e11c86c8ec1a5bc23b37482c436e8d08a4"}
Jul 17 11:48:51.268392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:51.267961    2566 scope.go:117] "RemoveContainer" containerID="369526013e96cd1cb0394f3806d0af22f1e5e1ffc8cd2d7aa9b71775067faf60"
Jul 17 11:48:51.268392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:51.268150    2566 scope.go:117] "RemoveContainer" containerID="a38d9b49c9691d7a44dd27e92e31f2e11c86c8ec1a5bc23b37482c436e8d08a4"
Jul 17 11:48:51.268503 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:51.268390    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm_odh-ai-gateway-infra(47aabcee-294b-465c-8240-fba6f722ec6a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm" podUID="47aabcee-294b-465c-8240-fba6f722ec6a"
Jul 17 11:48:51.269751 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:51.269736    2566 scope.go:117] "RemoveContainer" containerID="ff22e825e5c6fa03d85563bc9ea416a4b8037222e3355251fdd27148ef61699d"
Jul 17 11:48:51.269932 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:51.269915    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-d9458_odh-ai-gateway-infra(061abc49-5601-435f-9f81-aacc57d17653)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" podUID="061abc49-5601-435f-9f81-aacc57d17653"
Jul 17 11:48:52.277177 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:48:52.277153    2566 scope.go:117] "RemoveContainer" containerID="a38d9b49c9691d7a44dd27e92e31f2e11c86c8ec1a5bc23b37482c436e8d08a4"
Jul 17 11:48:52.277661 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:48:52.277419    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm_odh-ai-gateway-infra(47aabcee-294b-465c-8240-fba6f722ec6a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm" podUID="47aabcee-294b-465c-8240-fba6f722ec6a"
Jul 17 11:49:02.345723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:02.345673    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:49:03.651902 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:03.651866    2566 scope.go:117] "RemoveContainer" containerID="ff22e825e5c6fa03d85563bc9ea416a4b8037222e3355251fdd27148ef61699d"
Jul 17 11:49:03.652357 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:03.651970    2566 scope.go:117] "RemoveContainer" containerID="9de1125764a83499aee90508fa6c73972e952f4e4bda7c3e0bf709814fb3b2bf"
Jul 17 11:49:04.343586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:04.343562    2566 generic.go:356] "Generic (PLEG): container finished" podID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerID="42c79391f64ceec66d55994138ae123da12aad3e18bd6a3b1573ceecf94f4a94" exitCode=6
Jul 17 11:49:04.343679 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:04.343629    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb" event={"ID":"099f66a9-d185-4e34-acbb-f8b2b34770db","Type":"ContainerDied","Data":"42c79391f64ceec66d55994138ae123da12aad3e18bd6a3b1573ceecf94f4a94"}
Jul 17 11:49:04.343763 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:04.343681    2566 scope.go:117] "RemoveContainer" containerID="9de1125764a83499aee90508fa6c73972e952f4e4bda7c3e0bf709814fb3b2bf"
Jul 17 11:49:04.344033 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:04.344014    2566 scope.go:117] "RemoveContainer" containerID="42c79391f64ceec66d55994138ae123da12aad3e18bd6a3b1573ceecf94f4a94"
Jul 17 11:49:04.344276 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:04.344252    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-8cntb_odh-ai-gateway-infra(099f66a9-d185-4e34-acbb-f8b2b34770db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db"
Jul 17 11:49:04.345538 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:04.345519    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" event={"ID":"061abc49-5601-435f-9f81-aacc57d17653","Type":"ContainerStarted","Data":"07a79fdb9debfd4613027605e7bda31daff2ede5ba2d4d5b65eacbebccaac32d"}
Jul 17 11:49:04.345759 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:04.345746    2566 scope.go:117] "RemoveContainer" containerID="07a79fdb9debfd4613027605e7bda31daff2ede5ba2d4d5b65eacbebccaac32d"
Jul 17 11:49:04.345930 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:04.345914    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-d9458_odh-ai-gateway-infra(061abc49-5601-435f-9f81-aacc57d17653)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" podUID="061abc49-5601-435f-9f81-aacc57d17653"
Jul 17 11:49:05.353839 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.353803    2566 generic.go:356] "Generic (PLEG): container finished" podID="061abc49-5601-435f-9f81-aacc57d17653" containerID="07a79fdb9debfd4613027605e7bda31daff2ede5ba2d4d5b65eacbebccaac32d" exitCode=6
Jul 17 11:49:05.354220 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.353870    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" event={"ID":"061abc49-5601-435f-9f81-aacc57d17653","Type":"ContainerDied","Data":"07a79fdb9debfd4613027605e7bda31daff2ede5ba2d4d5b65eacbebccaac32d"}
Jul 17 11:49:05.354220 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.353956    2566 scope.go:117] "RemoveContainer" containerID="ff22e825e5c6fa03d85563bc9ea416a4b8037222e3355251fdd27148ef61699d"
Jul 17 11:49:05.354329 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.354236    2566 scope.go:117] "RemoveContainer" containerID="07a79fdb9debfd4613027605e7bda31daff2ede5ba2d4d5b65eacbebccaac32d"
Jul 17 11:49:05.354507 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:05.354477    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-d9458_odh-ai-gateway-infra(061abc49-5601-435f-9f81-aacc57d17653)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" podUID="061abc49-5601-435f-9f81-aacc57d17653"
Jul 17 11:49:05.372875 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.372852    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"]
Jul 17 11:49:05.390575 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.390434    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"]
Jul 17 11:49:05.506163 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.506141    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"
Jul 17 11:49:05.541843 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.541816    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099f66a9-d185-4e34-acbb-f8b2b34770db-maas-api-service-ca\") pod \"099f66a9-d185-4e34-acbb-f8b2b34770db\" (UID: \"099f66a9-d185-4e34-acbb-f8b2b34770db\") "
Jul 17 11:49:05.542154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.542130    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/099f66a9-d185-4e34-acbb-f8b2b34770db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "099f66a9-d185-4e34-acbb-f8b2b34770db" (UID: "099f66a9-d185-4e34-acbb-f8b2b34770db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:49:05.643101 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.643024    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099f66a9-d185-4e34-acbb-f8b2b34770db-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:49:05.652239 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:05.652219    2566 scope.go:117] "RemoveContainer" containerID="a38d9b49c9691d7a44dd27e92e31f2e11c86c8ec1a5bc23b37482c436e8d08a4"
Jul 17 11:49:06.360256 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.360227    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"
Jul 17 11:49:06.360671 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.360226    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb" event={"ID":"099f66a9-d185-4e34-acbb-f8b2b34770db","Type":"ContainerDied","Data":"e15600623c8a2dc7b2147dfdf5ab5bc51da21ff56f955a33b930e4dde6d0d78d"}
Jul 17 11:49:06.360671 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.360371    2566 scope.go:117] "RemoveContainer" containerID="42c79391f64ceec66d55994138ae123da12aad3e18bd6a3b1573ceecf94f4a94"
Jul 17 11:49:06.362342 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.362318    2566 generic.go:356] "Generic (PLEG): container finished" podID="47aabcee-294b-465c-8240-fba6f722ec6a" containerID="0b5ee73241cf099d9d23e6d454660bf4bf1074329a4571dfc692313c39be4621" exitCode=6
Jul 17 11:49:06.362495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.362387    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm" event={"ID":"47aabcee-294b-465c-8240-fba6f722ec6a","Type":"ContainerDied","Data":"0b5ee73241cf099d9d23e6d454660bf4bf1074329a4571dfc692313c39be4621"}
Jul 17 11:49:06.362751 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.362734    2566 scope.go:117] "RemoveContainer" containerID="0b5ee73241cf099d9d23e6d454660bf4bf1074329a4571dfc692313c39be4621"
Jul 17 11:49:06.362987 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:06.362963    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm_odh-ai-gateway-infra(47aabcee-294b-465c-8240-fba6f722ec6a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm" podUID="47aabcee-294b-465c-8240-fba6f722ec6a"
Jul 17 11:49:06.370379 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.370356    2566 scope.go:117] "RemoveContainer" containerID="a38d9b49c9691d7a44dd27e92e31f2e11c86c8ec1a5bc23b37482c436e8d08a4"
Jul 17 11:49:06.400652 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.400613    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"]
Jul 17 11:49:06.403118 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.403096    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8cntb"]
Jul 17 11:49:06.507949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.507925    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"
Jul 17 11:49:06.551067 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.551037    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/061abc49-5601-435f-9f81-aacc57d17653-maas-api-service-ca\") pod \"061abc49-5601-435f-9f81-aacc57d17653\" (UID: \"061abc49-5601-435f-9f81-aacc57d17653\") "
Jul 17 11:49:06.551352 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.551333    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/061abc49-5601-435f-9f81-aacc57d17653-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "061abc49-5601-435f-9f81-aacc57d17653" (UID: "061abc49-5601-435f-9f81-aacc57d17653"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:49:06.652430 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:06.652395    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/061abc49-5601-435f-9f81-aacc57d17653-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:49:07.370245 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.370214    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"
Jul 17 11:49:07.370744 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.370259    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458" event={"ID":"061abc49-5601-435f-9f81-aacc57d17653","Type":"ContainerDied","Data":"83e0d670be8a16d229e031e50840da602f073b65491a19d0f12458c48655b760"}
Jul 17 11:49:07.370744 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.370324    2566 scope.go:117] "RemoveContainer" containerID="07a79fdb9debfd4613027605e7bda31daff2ede5ba2d4d5b65eacbebccaac32d"
Jul 17 11:49:07.394112 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.394087    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"]
Jul 17 11:49:07.398067 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.398042    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"]
Jul 17 11:49:07.403916 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.403896    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-d9458"]
Jul 17 11:49:07.533663 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.533642    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"
Jul 17 11:49:07.558177 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.558150    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47aabcee-294b-465c-8240-fba6f722ec6a-maas-api-service-ca\") pod \"47aabcee-294b-465c-8240-fba6f722ec6a\" (UID: \"47aabcee-294b-465c-8240-fba6f722ec6a\") "
Jul 17 11:49:07.558477 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.558456    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47aabcee-294b-465c-8240-fba6f722ec6a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "47aabcee-294b-465c-8240-fba6f722ec6a" (UID: "47aabcee-294b-465c-8240-fba6f722ec6a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:49:07.658576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.658489    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="061abc49-5601-435f-9f81-aacc57d17653" path="/var/lib/kubelet/pods/061abc49-5601-435f-9f81-aacc57d17653/volumes"
Jul 17 11:49:07.658855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.658840    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" path="/var/lib/kubelet/pods/099f66a9-d185-4e34-acbb-f8b2b34770db/volumes"
Jul 17 11:49:07.658964 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:07.658944    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47aabcee-294b-465c-8240-fba6f722ec6a-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:49:08.385273 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:08.380245    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm" event={"ID":"47aabcee-294b-465c-8240-fba6f722ec6a","Type":"ContainerDied","Data":"ecaf8bbd9bc116e22056cf1f2c76069470db6e9b838f48e6f7dd0c66b70f7d62"}
Jul 17 11:49:08.385273 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:08.380367    2566 scope.go:117] "RemoveContainer" containerID="0b5ee73241cf099d9d23e6d454660bf4bf1074329a4571dfc692313c39be4621"
Jul 17 11:49:08.385273 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:08.385265    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"
Jul 17 11:49:08.407196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:08.407168    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"]
Jul 17 11:49:08.412693 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:08.412671    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-zdghm"]
Jul 17 11:49:09.203626 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.203594    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"]
Jul 17 11:49:09.203975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.203962    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="061abc49-5601-435f-9f81-aacc57d17653" containerName="revoke-keys"
Jul 17 11:49:09.204025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.203977    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="061abc49-5601-435f-9f81-aacc57d17653" containerName="revoke-keys"
Jul 17 11:49:09.204025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.203989    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerName="revoke-keys"
Jul 17 11:49:09.204025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.203995    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerName="revoke-keys"
Jul 17 11:49:09.204025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204001    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" containerName="revoke-keys"
Jul 17 11:49:09.204025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204007    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" containerName="revoke-keys"
Jul 17 11:49:09.204025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204018    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" containerName="revoke-keys"
Jul 17 11:49:09.204025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204023    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204032    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="061abc49-5601-435f-9f81-aacc57d17653" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204037    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="061abc49-5601-435f-9f81-aacc57d17653" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204044    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204048    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204059    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204064    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204114    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="061abc49-5601-435f-9f81-aacc57d17653" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204120    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="061abc49-5601-435f-9f81-aacc57d17653" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204126    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204131    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204137    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="061abc49-5601-435f-9f81-aacc57d17653" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204143    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204147    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerName="revoke-keys"
Jul 17 11:49:09.204238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.204155    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerName="revoke-keys"
Jul 17 11:49:09.207089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.207074    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"
Jul 17 11:49:09.209898 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.209879    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:49:09.210037 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.209918    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:49:09.215431 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.215408    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"]
Jul 17 11:49:09.276349 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.276305    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm\" (UID: \"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"
Jul 17 11:49:09.377753 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.377700    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm\" (UID: \"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"
Jul 17 11:49:09.378289 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.378272    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm\" (UID: \"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"
Jul 17 11:49:09.518424 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.518348    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"
Jul 17 11:49:09.651259 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.651231    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"]
Jul 17 11:49:09.652587 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:49:09.652560    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbeb106_45fb_4d7b_a7a7_5c2b0fc1244d.slice/crio-ae1881e3140cd13dc4d9920b10d04ca6bc30faaabd5edec0543aadc3b1a3bfe0 WatchSource:0}: Error finding container ae1881e3140cd13dc4d9920b10d04ca6bc30faaabd5edec0543aadc3b1a3bfe0: Status 404 returned error can't find the container with id ae1881e3140cd13dc4d9920b10d04ca6bc30faaabd5edec0543aadc3b1a3bfe0
Jul 17 11:49:09.658697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:09.658653    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" path="/var/lib/kubelet/pods/47aabcee-294b-465c-8240-fba6f722ec6a/volumes"
Jul 17 11:49:10.403112 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:10.403081    2566 generic.go:356] "Generic (PLEG): container finished" podID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerID="93b6e542071adddb933741b60a956be52ccadf90498e779b593ffbb2ac7cf927" exitCode=6
Jul 17 11:49:10.403264 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:10.403124    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm" event={"ID":"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d","Type":"ContainerDied","Data":"93b6e542071adddb933741b60a956be52ccadf90498e779b593ffbb2ac7cf927"}
Jul 17 11:49:10.403264 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:10.403172    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm" event={"ID":"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d","Type":"ContainerStarted","Data":"ae1881e3140cd13dc4d9920b10d04ca6bc30faaabd5edec0543aadc3b1a3bfe0"}
Jul 17 11:49:10.403380 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:10.403368    2566 scope.go:117] "RemoveContainer" containerID="93b6e542071adddb933741b60a956be52ccadf90498e779b593ffbb2ac7cf927"
Jul 17 11:49:11.410236 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:11.410138    2566 generic.go:356] "Generic (PLEG): container finished" podID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerID="0fb4a8571909a40f4e4722b410ad9d480612aa136b319b63e8e509e278509372" exitCode=6
Jul 17 11:49:11.410236 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:11.410182    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm" event={"ID":"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d","Type":"ContainerDied","Data":"0fb4a8571909a40f4e4722b410ad9d480612aa136b319b63e8e509e278509372"}
Jul 17 11:49:11.410236 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:11.410231    2566 scope.go:117] "RemoveContainer" containerID="93b6e542071adddb933741b60a956be52ccadf90498e779b593ffbb2ac7cf927"
Jul 17 11:49:11.410824 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:11.410463    2566 scope.go:117] "RemoveContainer" containerID="0fb4a8571909a40f4e4722b410ad9d480612aa136b319b63e8e509e278509372"
Jul 17 11:49:11.410824 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:11.410752    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm_odh-ai-gateway-infra(1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d"
Jul 17 11:49:12.416590 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:12.416563    2566 scope.go:117] "RemoveContainer" containerID="0fb4a8571909a40f4e4722b410ad9d480612aa136b319b63e8e509e278509372"
Jul 17 11:49:12.416989 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:12.416791    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm_odh-ai-gateway-infra(1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d"
Jul 17 11:49:19.240935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:19.240904    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:49:24.651725 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:24.651683    2566 scope.go:117] "RemoveContainer" containerID="0fb4a8571909a40f4e4722b410ad9d480612aa136b319b63e8e509e278509372"
Jul 17 11:49:25.484660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:25.484578    2566 generic.go:356] "Generic (PLEG): container finished" podID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerID="e1fdaee8c1c8841d46de98325a8b3a6c246c2e13d35c326bbdb68e7e37695a3d" exitCode=6
Jul 17 11:49:25.484660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:25.484648    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm" event={"ID":"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d","Type":"ContainerDied","Data":"e1fdaee8c1c8841d46de98325a8b3a6c246c2e13d35c326bbdb68e7e37695a3d"}
Jul 17 11:49:25.484869 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:25.484718    2566 scope.go:117] "RemoveContainer" containerID="0fb4a8571909a40f4e4722b410ad9d480612aa136b319b63e8e509e278509372"
Jul 17 11:49:25.485046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:25.485030    2566 scope.go:117] "RemoveContainer" containerID="e1fdaee8c1c8841d46de98325a8b3a6c246c2e13d35c326bbdb68e7e37695a3d"
Jul 17 11:49:25.485267 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:25.485242    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm_odh-ai-gateway-infra(1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d"
Jul 17 11:49:26.518534 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:26.517988    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"]
Jul 17 11:49:26.661492 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:26.661469    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"
Jul 17 11:49:26.725822 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:26.725794    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d-maas-api-service-ca\") pod \"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d\" (UID: \"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d\") "
Jul 17 11:49:26.726140 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:26.726105    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" (UID: "1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:49:26.726272 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:26.726211    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:49:27.498512 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:27.498478    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm" event={"ID":"1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d","Type":"ContainerDied","Data":"ae1881e3140cd13dc4d9920b10d04ca6bc30faaabd5edec0543aadc3b1a3bfe0"}
Jul 17 11:49:27.498512 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:27.498503    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"
Jul 17 11:49:27.498767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:27.498540    2566 scope.go:117] "RemoveContainer" containerID="e1fdaee8c1c8841d46de98325a8b3a6c246c2e13d35c326bbdb68e7e37695a3d"
Jul 17 11:49:27.530554 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:27.530527    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"]
Jul 17 11:49:27.536025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:27.536003    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-cgvrm"]
Jul 17 11:49:27.657933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:27.657905    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" path="/var/lib/kubelet/pods/1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d/volumes"
Jul 17 11:49:37.754787 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:37.754753    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:49:38.152144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152111    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"]
Jul 17 11:49:38.152462 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152451    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerName="revoke-keys"
Jul 17 11:49:38.152513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152464    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerName="revoke-keys"
Jul 17 11:49:38.152513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152477    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerName="revoke-keys"
Jul 17 11:49:38.152513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152484    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerName="revoke-keys"
Jul 17 11:49:38.152513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152498    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="061abc49-5601-435f-9f81-aacc57d17653" containerName="revoke-keys"
Jul 17 11:49:38.152513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152504    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="061abc49-5601-435f-9f81-aacc57d17653" containerName="revoke-keys"
Jul 17 11:49:38.152513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152511    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerName="revoke-keys"
Jul 17 11:49:38.152513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152516    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="099f66a9-d185-4e34-acbb-f8b2b34770db" containerName="revoke-keys"
Jul 17 11:49:38.152763 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152569    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="47aabcee-294b-465c-8240-fba6f722ec6a" containerName="revoke-keys"
Jul 17 11:49:38.152763 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152578    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerName="revoke-keys"
Jul 17 11:49:38.152763 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.152585    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerName="revoke-keys"
Jul 17 11:49:38.157030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.157015    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"
Jul 17 11:49:38.160300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.160278    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:49:38.160603 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.160585    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:49:38.165131 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.165112    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"]
Jul 17 11:49:38.315576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.315538    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e7e31-e781-404b-a38e-ee2c794135be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-8mxbh\" (UID: \"502e7e31-e781-404b-a38e-ee2c794135be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"
Jul 17 11:49:38.416097 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.416023    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e7e31-e781-404b-a38e-ee2c794135be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-8mxbh\" (UID: \"502e7e31-e781-404b-a38e-ee2c794135be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"
Jul 17 11:49:38.416639 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.416619    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e7e31-e781-404b-a38e-ee2c794135be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-8mxbh\" (UID: \"502e7e31-e781-404b-a38e-ee2c794135be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"
Jul 17 11:49:38.468833 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.468804    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"
Jul 17 11:49:38.598817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:38.598788    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"]
Jul 17 11:49:38.600849 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:49:38.600821    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502e7e31_e781_404b_a38e_ee2c794135be.slice/crio-27daf203f22eb74c30729bc756c9fe2d6dfb7386aaefb8d737e58f1fef85c6be WatchSource:0}: Error finding container 27daf203f22eb74c30729bc756c9fe2d6dfb7386aaefb8d737e58f1fef85c6be: Status 404 returned error can't find the container with id 27daf203f22eb74c30729bc756c9fe2d6dfb7386aaefb8d737e58f1fef85c6be
Jul 17 11:49:39.119081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.119012    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"]
Jul 17 11:49:39.119561 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.119542    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerName="revoke-keys"
Jul 17 11:49:39.119642 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.119564    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerName="revoke-keys"
Jul 17 11:49:39.119697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.119680    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cbeb106-45fb-4d7b-a7a7-5c2b0fc1244d" containerName="revoke-keys"
Jul 17 11:49:39.122604 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.122585    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"
Jul 17 11:49:39.132955 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.132935    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"]
Jul 17 11:49:39.222638 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.222611    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13825896-de67-4c83-8e24-52764d8408d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92\" (UID: \"13825896-de67-4c83-8e24-52764d8408d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"
Jul 17 11:49:39.323483 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.323455    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13825896-de67-4c83-8e24-52764d8408d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92\" (UID: \"13825896-de67-4c83-8e24-52764d8408d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"
Jul 17 11:49:39.324084 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.324066    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13825896-de67-4c83-8e24-52764d8408d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92\" (UID: \"13825896-de67-4c83-8e24-52764d8408d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"
Jul 17 11:49:39.432942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.432883    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"
Jul 17 11:49:39.563971 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.563948    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"]
Jul 17 11:49:39.565729 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:49:39.565685    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13825896_de67_4c83_8e24_52764d8408d6.slice/crio-ae6b880b14c91d0d64e4346aa70a1adee4ab8f5dd68f40c40112500afdb3ff52 WatchSource:0}: Error finding container ae6b880b14c91d0d64e4346aa70a1adee4ab8f5dd68f40c40112500afdb3ff52: Status 404 returned error can't find the container with id ae6b880b14c91d0d64e4346aa70a1adee4ab8f5dd68f40c40112500afdb3ff52
Jul 17 11:49:39.566206 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.566183    2566 generic.go:356] "Generic (PLEG): container finished" podID="502e7e31-e781-404b-a38e-ee2c794135be" containerID="9deab646e755c6a25c99e4f1539a816ad44cf10dde5e268a5c8d585374e28637" exitCode=6
Jul 17 11:49:39.566294 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.566262    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh" event={"ID":"502e7e31-e781-404b-a38e-ee2c794135be","Type":"ContainerDied","Data":"9deab646e755c6a25c99e4f1539a816ad44cf10dde5e268a5c8d585374e28637"}
Jul 17 11:49:39.566343 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.566305    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh" event={"ID":"502e7e31-e781-404b-a38e-ee2c794135be","Type":"ContainerStarted","Data":"27daf203f22eb74c30729bc756c9fe2d6dfb7386aaefb8d737e58f1fef85c6be"}
Jul 17 11:49:39.566472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:39.566458    2566 scope.go:117] "RemoveContainer" containerID="9deab646e755c6a25c99e4f1539a816ad44cf10dde5e268a5c8d585374e28637"
Jul 17 11:49:40.574721 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:40.574677    2566 generic.go:356] "Generic (PLEG): container finished" podID="502e7e31-e781-404b-a38e-ee2c794135be" containerID="9db8e21d8409d3f117b635bd81a775f17f933d3fa77cdf3f6d305c5547db42cd" exitCode=6
Jul 17 11:49:40.575192 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:40.574798    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh" event={"ID":"502e7e31-e781-404b-a38e-ee2c794135be","Type":"ContainerDied","Data":"9db8e21d8409d3f117b635bd81a775f17f933d3fa77cdf3f6d305c5547db42cd"}
Jul 17 11:49:40.575192 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:40.574892    2566 scope.go:117] "RemoveContainer" containerID="9deab646e755c6a25c99e4f1539a816ad44cf10dde5e268a5c8d585374e28637"
Jul 17 11:49:40.575192 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:40.575075    2566 scope.go:117] "RemoveContainer" containerID="9db8e21d8409d3f117b635bd81a775f17f933d3fa77cdf3f6d305c5547db42cd"
Jul 17 11:49:40.575371 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:40.575347    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-8mxbh_odh-ai-gateway-infra(502e7e31-e781-404b-a38e-ee2c794135be)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh" podUID="502e7e31-e781-404b-a38e-ee2c794135be"
Jul 17 11:49:40.576543 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:40.576524    2566 generic.go:356] "Generic (PLEG): container finished" podID="13825896-de67-4c83-8e24-52764d8408d6" containerID="7c407b4f27711d7f94864a919b8ca84c09b9521cc74b8ce389a0a349bac431c5" exitCode=6
Jul 17 11:49:40.576633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:40.576560    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92" event={"ID":"13825896-de67-4c83-8e24-52764d8408d6","Type":"ContainerDied","Data":"7c407b4f27711d7f94864a919b8ca84c09b9521cc74b8ce389a0a349bac431c5"}
Jul 17 11:49:40.576633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:40.576598    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92" event={"ID":"13825896-de67-4c83-8e24-52764d8408d6","Type":"ContainerStarted","Data":"ae6b880b14c91d0d64e4346aa70a1adee4ab8f5dd68f40c40112500afdb3ff52"}
Jul 17 11:49:40.576810 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:40.576798    2566 scope.go:117] "RemoveContainer" containerID="7c407b4f27711d7f94864a919b8ca84c09b9521cc74b8ce389a0a349bac431c5"
Jul 17 11:49:41.583811 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:41.583782    2566 generic.go:356] "Generic (PLEG): container finished" podID="13825896-de67-4c83-8e24-52764d8408d6" containerID="7f347e438932922a9fc6a665bc7b31cd4859f2fe3c1cf3fbb7c6ab42285edd5c" exitCode=6
Jul 17 11:49:41.584277 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:41.583865    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92" event={"ID":"13825896-de67-4c83-8e24-52764d8408d6","Type":"ContainerDied","Data":"7f347e438932922a9fc6a665bc7b31cd4859f2fe3c1cf3fbb7c6ab42285edd5c"}
Jul 17 11:49:41.584277 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:41.583915    2566 scope.go:117] "RemoveContainer" containerID="7c407b4f27711d7f94864a919b8ca84c09b9521cc74b8ce389a0a349bac431c5"
Jul 17 11:49:41.584277 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:41.584157    2566 scope.go:117] "RemoveContainer" containerID="7f347e438932922a9fc6a665bc7b31cd4859f2fe3c1cf3fbb7c6ab42285edd5c"
Jul 17 11:49:41.584447 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:41.584416    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92_odh-ai-gateway-infra(13825896-de67-4c83-8e24-52764d8408d6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92" podUID="13825896-de67-4c83-8e24-52764d8408d6"
Jul 17 11:49:41.585737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:41.585699    2566 scope.go:117] "RemoveContainer" containerID="9db8e21d8409d3f117b635bd81a775f17f933d3fa77cdf3f6d305c5547db42cd"
Jul 17 11:49:41.585936 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:41.585918    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-8mxbh_odh-ai-gateway-infra(502e7e31-e781-404b-a38e-ee2c794135be)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh" podUID="502e7e31-e781-404b-a38e-ee2c794135be"
Jul 17 11:49:42.598996 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:42.598970    2566 scope.go:117] "RemoveContainer" containerID="7f347e438932922a9fc6a665bc7b31cd4859f2fe3c1cf3fbb7c6ab42285edd5c"
Jul 17 11:49:42.599358 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:42.599139    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92_odh-ai-gateway-infra(13825896-de67-4c83-8e24-52764d8408d6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92" podUID="13825896-de67-4c83-8e24-52764d8408d6"
Jul 17 11:49:45.960762 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:45.960722    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"]
Jul 17 11:49:45.965584 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:45.965565    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"
Jul 17 11:49:45.972284 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:45.972257    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"]
Jul 17 11:49:46.085655 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:46.085620    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/394726b9-3f5f-4f2e-8008-132fecf5a224-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-8s2s4\" (UID: \"394726b9-3f5f-4f2e-8008-132fecf5a224\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"
Jul 17 11:49:46.186607 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:46.186568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/394726b9-3f5f-4f2e-8008-132fecf5a224-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-8s2s4\" (UID: \"394726b9-3f5f-4f2e-8008-132fecf5a224\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"
Jul 17 11:49:46.187220 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:46.187199    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/394726b9-3f5f-4f2e-8008-132fecf5a224-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-8s2s4\" (UID: \"394726b9-3f5f-4f2e-8008-132fecf5a224\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"
Jul 17 11:49:46.276391 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:46.276314    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"
Jul 17 11:49:46.412826 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:46.412768    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"]
Jul 17 11:49:46.415070 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:49:46.415041    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod394726b9_3f5f_4f2e_8008_132fecf5a224.slice/crio-4d7193ae7447f0926278320fdcc2950bbd50116299c03e3d9b1862ac36f4af7f WatchSource:0}: Error finding container 4d7193ae7447f0926278320fdcc2950bbd50116299c03e3d9b1862ac36f4af7f: Status 404 returned error can't find the container with id 4d7193ae7447f0926278320fdcc2950bbd50116299c03e3d9b1862ac36f4af7f
Jul 17 11:49:46.623556 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:46.623526    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4" event={"ID":"394726b9-3f5f-4f2e-8008-132fecf5a224","Type":"ContainerStarted","Data":"4d7193ae7447f0926278320fdcc2950bbd50116299c03e3d9b1862ac36f4af7f"}
Jul 17 11:49:47.630057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:47.630024    2566 generic.go:356] "Generic (PLEG): container finished" podID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerID="f032cd3c6cc49a9cd2b536ddfa75fc73b427abc41defab5079afe7a5442822a7" exitCode=6
Jul 17 11:49:47.630550 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:47.630104    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4" event={"ID":"394726b9-3f5f-4f2e-8008-132fecf5a224","Type":"ContainerDied","Data":"f032cd3c6cc49a9cd2b536ddfa75fc73b427abc41defab5079afe7a5442822a7"}
Jul 17 11:49:47.630550 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:47.630326    2566 scope.go:117] "RemoveContainer" containerID="f032cd3c6cc49a9cd2b536ddfa75fc73b427abc41defab5079afe7a5442822a7"
Jul 17 11:49:48.636856 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:48.636823    2566 generic.go:356] "Generic (PLEG): container finished" podID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerID="7c4c2834bb765ec85233dac110c74f0b753adc6f8769627544da98ae31aca0c1" exitCode=6
Jul 17 11:49:48.637314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:48.636880    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4" event={"ID":"394726b9-3f5f-4f2e-8008-132fecf5a224","Type":"ContainerDied","Data":"7c4c2834bb765ec85233dac110c74f0b753adc6f8769627544da98ae31aca0c1"}
Jul 17 11:49:48.637314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:48.636929    2566 scope.go:117] "RemoveContainer" containerID="f032cd3c6cc49a9cd2b536ddfa75fc73b427abc41defab5079afe7a5442822a7"
Jul 17 11:49:48.637314 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:48.637094    2566 scope.go:117] "RemoveContainer" containerID="7c4c2834bb765ec85233dac110c74f0b753adc6f8769627544da98ae31aca0c1"
Jul 17 11:49:48.637446 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:48.637328    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-8s2s4_odh-ai-gateway-infra(394726b9-3f5f-4f2e-8008-132fecf5a224)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224"
Jul 17 11:49:49.643413 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:49.643387    2566 scope.go:117] "RemoveContainer" containerID="7c4c2834bb765ec85233dac110c74f0b753adc6f8769627544da98ae31aca0c1"
Jul 17 11:49:49.643795 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:49.643585    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-8s2s4_odh-ai-gateway-infra(394726b9-3f5f-4f2e-8008-132fecf5a224)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224"
Jul 17 11:49:52.652163 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:52.652121    2566 scope.go:117] "RemoveContainer" containerID="7f347e438932922a9fc6a665bc7b31cd4859f2fe3c1cf3fbb7c6ab42285edd5c"
Jul 17 11:49:53.664411 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:53.664378    2566 generic.go:356] "Generic (PLEG): container finished" podID="13825896-de67-4c83-8e24-52764d8408d6" containerID="c04bf6169ec6049e545023e7c8f132ee38c65707d95ba02f30bac71317cd67eb" exitCode=6
Jul 17 11:49:53.664871 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:53.664453    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92" event={"ID":"13825896-de67-4c83-8e24-52764d8408d6","Type":"ContainerDied","Data":"c04bf6169ec6049e545023e7c8f132ee38c65707d95ba02f30bac71317cd67eb"}
Jul 17 11:49:53.664871 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:53.664508    2566 scope.go:117] "RemoveContainer" containerID="7f347e438932922a9fc6a665bc7b31cd4859f2fe3c1cf3fbb7c6ab42285edd5c"
Jul 17 11:49:53.664871 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:53.664777    2566 scope.go:117] "RemoveContainer" containerID="c04bf6169ec6049e545023e7c8f132ee38c65707d95ba02f30bac71317cd67eb"
Jul 17 11:49:53.665053 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:53.664978    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92_odh-ai-gateway-infra(13825896-de67-4c83-8e24-52764d8408d6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92" podUID="13825896-de67-4c83-8e24-52764d8408d6"
Jul 17 11:49:54.706555 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:54.701889    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"]
Jul 17 11:49:54.857443 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:54.857415    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"
Jul 17 11:49:54.859567 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:54.859546    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13825896-de67-4c83-8e24-52764d8408d6-maas-api-service-ca\") pod \"13825896-de67-4c83-8e24-52764d8408d6\" (UID: \"13825896-de67-4c83-8e24-52764d8408d6\") "
Jul 17 11:49:54.859916 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:54.859899    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13825896-de67-4c83-8e24-52764d8408d6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "13825896-de67-4c83-8e24-52764d8408d6" (UID: "13825896-de67-4c83-8e24-52764d8408d6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:49:54.960931 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:54.960854    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13825896-de67-4c83-8e24-52764d8408d6-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:49:55.677778 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:55.677746    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92" event={"ID":"13825896-de67-4c83-8e24-52764d8408d6","Type":"ContainerDied","Data":"ae6b880b14c91d0d64e4346aa70a1adee4ab8f5dd68f40c40112500afdb3ff52"}
Jul 17 11:49:55.677914 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:55.677808    2566 scope.go:117] "RemoveContainer" containerID="c04bf6169ec6049e545023e7c8f132ee38c65707d95ba02f30bac71317cd67eb"
Jul 17 11:49:55.677914 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:55.677759    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"
Jul 17 11:49:55.699319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:55.699286    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"]
Jul 17 11:49:55.705844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:55.705821    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mvl92"]
Jul 17 11:49:56.645064 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:56.645034    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:49:56.651765 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:56.651735    2566 scope.go:117] "RemoveContainer" containerID="9db8e21d8409d3f117b635bd81a775f17f933d3fa77cdf3f6d305c5547db42cd"
Jul 17 11:49:57.663384 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:57.663354    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13825896-de67-4c83-8e24-52764d8408d6" path="/var/lib/kubelet/pods/13825896-de67-4c83-8e24-52764d8408d6/volumes"
Jul 17 11:49:57.691369 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:57.691339    2566 generic.go:356] "Generic (PLEG): container finished" podID="502e7e31-e781-404b-a38e-ee2c794135be" containerID="dd384ff85d3787e364137a56364b62e17cb8717b42775078868e028e7c8ee5d3" exitCode=6
Jul 17 11:49:57.691564 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:57.691373    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh" event={"ID":"502e7e31-e781-404b-a38e-ee2c794135be","Type":"ContainerDied","Data":"dd384ff85d3787e364137a56364b62e17cb8717b42775078868e028e7c8ee5d3"}
Jul 17 11:49:57.691564 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:57.691416    2566 scope.go:117] "RemoveContainer" containerID="9db8e21d8409d3f117b635bd81a775f17f933d3fa77cdf3f6d305c5547db42cd"
Jul 17 11:49:57.691721 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:57.691687    2566 scope.go:117] "RemoveContainer" containerID="dd384ff85d3787e364137a56364b62e17cb8717b42775078868e028e7c8ee5d3"
Jul 17 11:49:57.691950 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:49:57.691928    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-8mxbh_odh-ai-gateway-infra(502e7e31-e781-404b-a38e-ee2c794135be)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh" podUID="502e7e31-e781-404b-a38e-ee2c794135be"
Jul 17 11:49:58.720905 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:58.720867    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"]
Jul 17 11:49:58.847376 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:58.847354    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"
Jul 17 11:49:58.894167 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:58.894136    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e7e31-e781-404b-a38e-ee2c794135be-maas-api-service-ca\") pod \"502e7e31-e781-404b-a38e-ee2c794135be\" (UID: \"502e7e31-e781-404b-a38e-ee2c794135be\") "
Jul 17 11:49:58.894472 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:58.894426    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/502e7e31-e781-404b-a38e-ee2c794135be-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "502e7e31-e781-404b-a38e-ee2c794135be" (UID: "502e7e31-e781-404b-a38e-ee2c794135be"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:49:58.995295 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:58.995221    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e7e31-e781-404b-a38e-ee2c794135be-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:49:59.260325 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260238    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"]
Jul 17 11:49:59.260687 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260670    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="502e7e31-e781-404b-a38e-ee2c794135be" containerName="revoke-keys"
Jul 17 11:49:59.260815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260690    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="502e7e31-e781-404b-a38e-ee2c794135be" containerName="revoke-keys"
Jul 17 11:49:59.260815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260733    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13825896-de67-4c83-8e24-52764d8408d6" containerName="revoke-keys"
Jul 17 11:49:59.260815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260742    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="13825896-de67-4c83-8e24-52764d8408d6" containerName="revoke-keys"
Jul 17 11:49:59.260815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260757    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="502e7e31-e781-404b-a38e-ee2c794135be" containerName="revoke-keys"
Jul 17 11:49:59.260815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260767    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="502e7e31-e781-404b-a38e-ee2c794135be" containerName="revoke-keys"
Jul 17 11:49:59.260815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260783    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13825896-de67-4c83-8e24-52764d8408d6" containerName="revoke-keys"
Jul 17 11:49:59.260815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260792    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="13825896-de67-4c83-8e24-52764d8408d6" containerName="revoke-keys"
Jul 17 11:49:59.260815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260803    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13825896-de67-4c83-8e24-52764d8408d6" containerName="revoke-keys"
Jul 17 11:49:59.260815 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260811    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="13825896-de67-4c83-8e24-52764d8408d6" containerName="revoke-keys"
Jul 17 11:49:59.261257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260822    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="502e7e31-e781-404b-a38e-ee2c794135be" containerName="revoke-keys"
Jul 17 11:49:59.261257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260830    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="502e7e31-e781-404b-a38e-ee2c794135be" containerName="revoke-keys"
Jul 17 11:49:59.261257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260937    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="502e7e31-e781-404b-a38e-ee2c794135be" containerName="revoke-keys"
Jul 17 11:49:59.261257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260950    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="502e7e31-e781-404b-a38e-ee2c794135be" containerName="revoke-keys"
Jul 17 11:49:59.261257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260961    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="502e7e31-e781-404b-a38e-ee2c794135be" containerName="revoke-keys"
Jul 17 11:49:59.261257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260971    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="13825896-de67-4c83-8e24-52764d8408d6" containerName="revoke-keys"
Jul 17 11:49:59.261257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260980    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="13825896-de67-4c83-8e24-52764d8408d6" containerName="revoke-keys"
Jul 17 11:49:59.261257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.260988    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="13825896-de67-4c83-8e24-52764d8408d6" containerName="revoke-keys"
Jul 17 11:49:59.264269 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.264248    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"
Jul 17 11:49:59.271830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.271810    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"]
Jul 17 11:49:59.298036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.298010    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabfa95-34a3-46a0-be63-53a1d430a017-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx\" (UID: \"ebabfa95-34a3-46a0-be63-53a1d430a017\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"
Jul 17 11:49:59.398793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.398767    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabfa95-34a3-46a0-be63-53a1d430a017-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx\" (UID: \"ebabfa95-34a3-46a0-be63-53a1d430a017\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"
Jul 17 11:49:59.399328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.399304    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabfa95-34a3-46a0-be63-53a1d430a017-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx\" (UID: \"ebabfa95-34a3-46a0-be63-53a1d430a017\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"
Jul 17 11:49:59.575873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.575848    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"
Jul 17 11:49:59.704395 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.704371    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"
Jul 17 11:49:59.704545 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.704369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh" event={"ID":"502e7e31-e781-404b-a38e-ee2c794135be","Type":"ContainerDied","Data":"27daf203f22eb74c30729bc756c9fe2d6dfb7386aaefb8d737e58f1fef85c6be"}
Jul 17 11:49:59.704545 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.704513    2566 scope.go:117] "RemoveContainer" containerID="dd384ff85d3787e364137a56364b62e17cb8717b42775078868e028e7c8ee5d3"
Jul 17 11:49:59.712365 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.712342    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"]
Jul 17 11:49:59.715626 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:49:59.715605    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebabfa95_34a3_46a0_be63_53a1d430a017.slice/crio-b268b404944405a4924997721c4a22e91af78faf29f7154deadb40f1a7ce804f WatchSource:0}: Error finding container b268b404944405a4924997721c4a22e91af78faf29f7154deadb40f1a7ce804f: Status 404 returned error can't find the container with id b268b404944405a4924997721c4a22e91af78faf29f7154deadb40f1a7ce804f
Jul 17 11:49:59.724645 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.724624    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"]
Jul 17 11:49:59.730579 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:49:59.730560    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8mxbh"]
Jul 17 11:50:00.652422 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:00.652345    2566 scope.go:117] "RemoveContainer" containerID="7c4c2834bb765ec85233dac110c74f0b753adc6f8769627544da98ae31aca0c1"
Jul 17 11:50:00.712044 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:00.712023    2566 generic.go:356] "Generic (PLEG): container finished" podID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerID="e368aea6e2fad8e9b28456f5be29de2e96453e8fc820fcb436fec5087fdea5de" exitCode=6
Jul 17 11:50:00.712144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:00.712051    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx" event={"ID":"ebabfa95-34a3-46a0-be63-53a1d430a017","Type":"ContainerDied","Data":"e368aea6e2fad8e9b28456f5be29de2e96453e8fc820fcb436fec5087fdea5de"}
Jul 17 11:50:00.712144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:00.712090    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx" event={"ID":"ebabfa95-34a3-46a0-be63-53a1d430a017","Type":"ContainerStarted","Data":"b268b404944405a4924997721c4a22e91af78faf29f7154deadb40f1a7ce804f"}
Jul 17 11:50:00.712218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:00.712208    2566 scope.go:117] "RemoveContainer" containerID="e368aea6e2fad8e9b28456f5be29de2e96453e8fc820fcb436fec5087fdea5de"
Jul 17 11:50:01.658359 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:01.658283    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="502e7e31-e781-404b-a38e-ee2c794135be" path="/var/lib/kubelet/pods/502e7e31-e781-404b-a38e-ee2c794135be/volumes"
Jul 17 11:50:01.724495 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:01.724461    2566 generic.go:356] "Generic (PLEG): container finished" podID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerID="552f9290c14930a539299d4c1e0e0760f6ab6215fb0da17d30cf3a9e896be751" exitCode=6
Jul 17 11:50:01.724655 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:01.724543    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx" event={"ID":"ebabfa95-34a3-46a0-be63-53a1d430a017","Type":"ContainerDied","Data":"552f9290c14930a539299d4c1e0e0760f6ab6215fb0da17d30cf3a9e896be751"}
Jul 17 11:50:01.724655 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:01.724636    2566 scope.go:117] "RemoveContainer" containerID="e368aea6e2fad8e9b28456f5be29de2e96453e8fc820fcb436fec5087fdea5de"
Jul 17 11:50:01.724814 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:01.724738    2566 scope.go:117] "RemoveContainer" containerID="552f9290c14930a539299d4c1e0e0760f6ab6215fb0da17d30cf3a9e896be751"
Jul 17 11:50:01.724972 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:01.724953    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx_odh-ai-gateway-infra(ebabfa95-34a3-46a0-be63-53a1d430a017)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017"
Jul 17 11:50:01.726493 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:01.726475    2566 generic.go:356] "Generic (PLEG): container finished" podID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerID="7737e8b8802fe47dcc03b804ce213d1e34c3686c0a18f2975179c8d67298faf6" exitCode=6
Jul 17 11:50:01.726571 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:01.726542    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4" event={"ID":"394726b9-3f5f-4f2e-8008-132fecf5a224","Type":"ContainerDied","Data":"7737e8b8802fe47dcc03b804ce213d1e34c3686c0a18f2975179c8d67298faf6"}
Jul 17 11:50:01.726828 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:01.726811    2566 scope.go:117] "RemoveContainer" containerID="7737e8b8802fe47dcc03b804ce213d1e34c3686c0a18f2975179c8d67298faf6"
Jul 17 11:50:01.726979 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:01.726961    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-8s2s4_odh-ai-gateway-infra(394726b9-3f5f-4f2e-8008-132fecf5a224)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224"
Jul 17 11:50:01.735900 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:01.735882    2566 scope.go:117] "RemoveContainer" containerID="7c4c2834bb765ec85233dac110c74f0b753adc6f8769627544da98ae31aca0c1"
Jul 17 11:50:02.734276 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:02.734250    2566 scope.go:117] "RemoveContainer" containerID="552f9290c14930a539299d4c1e0e0760f6ab6215fb0da17d30cf3a9e896be751"
Jul 17 11:50:02.734722 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:02.734515    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx_odh-ai-gateway-infra(ebabfa95-34a3-46a0-be63-53a1d430a017)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017"
Jul 17 11:50:02.771191 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:02.771166    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"]
Jul 17 11:50:02.904869 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:02.904847    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"
Jul 17 11:50:02.925271 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:02.925243    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/394726b9-3f5f-4f2e-8008-132fecf5a224-maas-api-service-ca\") pod \"394726b9-3f5f-4f2e-8008-132fecf5a224\" (UID: \"394726b9-3f5f-4f2e-8008-132fecf5a224\") "
Jul 17 11:50:02.925656 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:02.925631    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/394726b9-3f5f-4f2e-8008-132fecf5a224-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "394726b9-3f5f-4f2e-8008-132fecf5a224" (UID: "394726b9-3f5f-4f2e-8008-132fecf5a224"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:50:03.026540 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:03.026465    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/394726b9-3f5f-4f2e-8008-132fecf5a224-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:50:03.741125 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:03.741103    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"
Jul 17 11:50:03.741125 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:03.741110    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4" event={"ID":"394726b9-3f5f-4f2e-8008-132fecf5a224","Type":"ContainerDied","Data":"4d7193ae7447f0926278320fdcc2950bbd50116299c03e3d9b1862ac36f4af7f"}
Jul 17 11:50:03.741580 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:03.741162    2566 scope.go:117] "RemoveContainer" containerID="7737e8b8802fe47dcc03b804ce213d1e34c3686c0a18f2975179c8d67298faf6"
Jul 17 11:50:03.761139 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:03.761116    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"]
Jul 17 11:50:03.767844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:03.767823    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-8s2s4"]
Jul 17 11:50:05.658169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:05.658139    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" path="/var/lib/kubelet/pods/394726b9-3f5f-4f2e-8008-132fecf5a224/volumes"
Jul 17 11:50:11.536166 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:11.536133    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:50:17.654626 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:17.654591    2566 scope.go:117] "RemoveContainer" containerID="552f9290c14930a539299d4c1e0e0760f6ab6215fb0da17d30cf3a9e896be751"
Jul 17 11:50:18.823072 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.823044    2566 generic.go:356] "Generic (PLEG): container finished" podID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerID="a78d6b6f973c49ecb0bbbcdee54e4bd41b550bf0f688cd1bfe9dab7c6c274b02" exitCode=6
Jul 17 11:50:18.823456 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.823094    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx" event={"ID":"ebabfa95-34a3-46a0-be63-53a1d430a017","Type":"ContainerDied","Data":"a78d6b6f973c49ecb0bbbcdee54e4bd41b550bf0f688cd1bfe9dab7c6c274b02"}
Jul 17 11:50:18.823456 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.823140    2566 scope.go:117] "RemoveContainer" containerID="552f9290c14930a539299d4c1e0e0760f6ab6215fb0da17d30cf3a9e896be751"
Jul 17 11:50:18.823593 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.823460    2566 scope.go:117] "RemoveContainer" containerID="a78d6b6f973c49ecb0bbbcdee54e4bd41b550bf0f688cd1bfe9dab7c6c274b02"
Jul 17 11:50:18.823663 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:18.823643    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx_odh-ai-gateway-infra(ebabfa95-34a3-46a0-be63-53a1d430a017)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017"
Jul 17 11:50:18.897772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.897731    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"]
Jul 17 11:50:18.898161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.898137    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerName="revoke-keys"
Jul 17 11:50:18.898161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.898154    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerName="revoke-keys"
Jul 17 11:50:18.898161 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.898163    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerName="revoke-keys"
Jul 17 11:50:18.898395 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.898171    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerName="revoke-keys"
Jul 17 11:50:18.898395 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.898200    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerName="revoke-keys"
Jul 17 11:50:18.898395 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.898207    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerName="revoke-keys"
Jul 17 11:50:18.898395 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.898273    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerName="revoke-keys"
Jul 17 11:50:18.898395 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.898289    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerName="revoke-keys"
Jul 17 11:50:18.902370 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.902355    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:18.905379 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.905356    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ad4893\"/\"kube-root-ca.crt\""
Jul 17 11:50:18.905497 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.905397    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ad4893\"/\"openshift-service-ca.crt\""
Jul 17 11:50:18.905497 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.905414    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ad4893\"/\"default-dockercfg-9dnx7\""
Jul 17 11:50:18.905497 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.905445    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ad4893\"/\"sub-test-model-ad4893-kserve-self-signed-certs\""
Jul 17 11:50:18.910694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.910671    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"]
Jul 17 11:50:18.965436 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.965403    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-tmp-dir\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:18.965598 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.965454    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc5b3650-e267-438d-8a01-76dd7752723a-tls-certs\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:18.965598 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.965474    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqx77\" (UniqueName: \"kubernetes.io/projected/fc5b3650-e267-438d-8a01-76dd7752723a-kube-api-access-lqx77\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:18.965598 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.965531    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-model-cache\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:18.965738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.965616    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-home\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:18.965738 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:18.965642    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-dshm\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.066446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.066414    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-home\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.066446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.066445    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-dshm\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.066628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.066472    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-tmp-dir\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.066628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.066502    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc5b3650-e267-438d-8a01-76dd7752723a-tls-certs\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.066628 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.066609    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqx77\" (UniqueName: \"kubernetes.io/projected/fc5b3650-e267-438d-8a01-76dd7752723a-kube-api-access-lqx77\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.066788 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.066654    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-model-cache\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.066888 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.066864    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-home\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.066942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.066885    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-tmp-dir\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.066991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.066976    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-model-cache\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.068635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.068614    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-dshm\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.068943 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.068926    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc5b3650-e267-438d-8a01-76dd7752723a-tls-certs\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.075943 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.075888    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqx77\" (UniqueName: \"kubernetes.io/projected/fc5b3650-e267-438d-8a01-76dd7752723a-kube-api-access-lqx77\") pod \"sub-test-model-ad4893-kserve-698fb6869-cnk2j\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") " pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.212599 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.212570    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:19.344409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.344377    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"]
Jul 17 11:50:19.347045 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:50:19.347005    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc5b3650_e267_438d_8a01_76dd7752723a.slice/crio-8e57b235898d3dd8256db9a5fb18bb7e8a6ed982923f74d327728a2f4e124d15 WatchSource:0}: Error finding container 8e57b235898d3dd8256db9a5fb18bb7e8a6ed982923f74d327728a2f4e124d15: Status 404 returned error can't find the container with id 8e57b235898d3dd8256db9a5fb18bb7e8a6ed982923f74d327728a2f4e124d15
Jul 17 11:50:19.831963 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.831914    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j" event={"ID":"fc5b3650-e267-438d-8a01-76dd7752723a","Type":"ContainerStarted","Data":"34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc"}
Jul 17 11:50:19.832364 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.831971    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j" event={"ID":"fc5b3650-e267-438d-8a01-76dd7752723a","Type":"ContainerStarted","Data":"8e57b235898d3dd8256db9a5fb18bb7e8a6ed982923f74d327728a2f4e124d15"}
Jul 17 11:50:19.851327 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.851279    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j" podStartSLOduration=1.85126446 podStartE2EDuration="1.85126446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:50:19.848379099 +0000 UTC m=+3202.705463045" watchObservedRunningTime="2026-07-17 11:50:19.85126446 +0000 UTC m=+3202.708348458"
Jul 17 11:50:19.854067 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.854046    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"]
Jul 17 11:50:19.980841 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:19.980815    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"
Jul 17 11:50:20.076502 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:20.076474    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabfa95-34a3-46a0-be63-53a1d430a017-maas-api-service-ca\") pod \"ebabfa95-34a3-46a0-be63-53a1d430a017\" (UID: \"ebabfa95-34a3-46a0-be63-53a1d430a017\") "
Jul 17 11:50:20.076874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:20.076839    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebabfa95-34a3-46a0-be63-53a1d430a017-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ebabfa95-34a3-46a0-be63-53a1d430a017" (UID: "ebabfa95-34a3-46a0-be63-53a1d430a017"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:50:20.177981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:20.177907    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabfa95-34a3-46a0-be63-53a1d430a017-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:50:20.839016 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:20.838985    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"
Jul 17 11:50:20.839441 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:20.839035    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx" event={"ID":"ebabfa95-34a3-46a0-be63-53a1d430a017","Type":"ContainerDied","Data":"b268b404944405a4924997721c4a22e91af78faf29f7154deadb40f1a7ce804f"}
Jul 17 11:50:20.839441 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:20.839090    2566 scope.go:117] "RemoveContainer" containerID="a78d6b6f973c49ecb0bbbcdee54e4bd41b550bf0f688cd1bfe9dab7c6c274b02"
Jul 17 11:50:20.861811 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:20.861788    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"]
Jul 17 11:50:20.868439 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:20.868414    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-zhqlx"]
Jul 17 11:50:21.658677 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:21.658648    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" path="/var/lib/kubelet/pods/ebabfa95-34a3-46a0-be63-53a1d430a017/volumes"
Jul 17 11:50:29.165565 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.165528    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"]
Jul 17 11:50:29.166041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.165927    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerName="revoke-keys"
Jul 17 11:50:29.166041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.165940    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerName="revoke-keys"
Jul 17 11:50:29.166041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.165959    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerName="revoke-keys"
Jul 17 11:50:29.166041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.165965    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerName="revoke-keys"
Jul 17 11:50:29.166041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.165973    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerName="revoke-keys"
Jul 17 11:50:29.166041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.165979    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerName="revoke-keys"
Jul 17 11:50:29.166233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.166049    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerName="revoke-keys"
Jul 17 11:50:29.166233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.166059    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="394726b9-3f5f-4f2e-8008-132fecf5a224" containerName="revoke-keys"
Jul 17 11:50:29.166233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.166068    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerName="revoke-keys"
Jul 17 11:50:29.166233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.166074    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebabfa95-34a3-46a0-be63-53a1d430a017" containerName="revoke-keys"
Jul 17 11:50:29.170401 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.170381    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"
Jul 17 11:50:29.173371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.173331    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:50:29.173371 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.173362    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:50:29.177557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.177532    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"]
Jul 17 11:50:29.213609 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.213587    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:29.213697 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.213624    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:29.225657 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.225639    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:29.259418 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.259399    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da901a6e-8905-4050-9a5d-c81f9237f8f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv\" (UID: \"da901a6e-8905-4050-9a5d-c81f9237f8f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"
Jul 17 11:50:29.360203 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.360178    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da901a6e-8905-4050-9a5d-c81f9237f8f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv\" (UID: \"da901a6e-8905-4050-9a5d-c81f9237f8f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"
Jul 17 11:50:29.360795 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.360779    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da901a6e-8905-4050-9a5d-c81f9237f8f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv\" (UID: \"da901a6e-8905-4050-9a5d-c81f9237f8f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"
Jul 17 11:50:29.482540 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.482463    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"
Jul 17 11:50:29.610116 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.610089    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"]
Jul 17 11:50:29.611978 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:50:29.611952    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda901a6e_8905_4050_9a5d_c81f9237f8f6.slice/crio-79240dbefa980b254cf84b32507a97e995cd830624f0665da8d4332c7595e61b WatchSource:0}: Error finding container 79240dbefa980b254cf84b32507a97e995cd830624f0665da8d4332c7595e61b: Status 404 returned error can't find the container with id 79240dbefa980b254cf84b32507a97e995cd830624f0665da8d4332c7595e61b
Jul 17 11:50:29.886956 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.886931    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" event={"ID":"da901a6e-8905-4050-9a5d-c81f9237f8f6","Type":"ContainerStarted","Data":"79240dbefa980b254cf84b32507a97e995cd830624f0665da8d4332c7595e61b"}
Jul 17 11:50:29.898047 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:29.898024    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:50:30.894377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:30.894343    2566 generic.go:356] "Generic (PLEG): container finished" podID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerID="4fd0cdfa76b46da8e1406f73814548814eca17da2054731473ec757f68db76cc" exitCode=6
Jul 17 11:50:30.894895 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:30.894437    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" event={"ID":"da901a6e-8905-4050-9a5d-c81f9237f8f6","Type":"ContainerDied","Data":"4fd0cdfa76b46da8e1406f73814548814eca17da2054731473ec757f68db76cc"}
Jul 17 11:50:30.894895 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:30.894613    2566 scope.go:117] "RemoveContainer" containerID="4fd0cdfa76b46da8e1406f73814548814eca17da2054731473ec757f68db76cc"
Jul 17 11:50:31.901027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:31.900950    2566 generic.go:356] "Generic (PLEG): container finished" podID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerID="d76b203ac449c0de68ede17e585ad9fc2b6a57594688f9d2a85401acd5e067ad" exitCode=6
Jul 17 11:50:31.901400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:31.901036    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" event={"ID":"da901a6e-8905-4050-9a5d-c81f9237f8f6","Type":"ContainerDied","Data":"d76b203ac449c0de68ede17e585ad9fc2b6a57594688f9d2a85401acd5e067ad"}
Jul 17 11:50:31.901400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:31.901096    2566 scope.go:117] "RemoveContainer" containerID="4fd0cdfa76b46da8e1406f73814548814eca17da2054731473ec757f68db76cc"
Jul 17 11:50:31.901400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:31.901250    2566 scope.go:117] "RemoveContainer" containerID="d76b203ac449c0de68ede17e585ad9fc2b6a57594688f9d2a85401acd5e067ad"
Jul 17 11:50:31.901523 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:31.901473    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv_odh-ai-gateway-infra(da901a6e-8905-4050-9a5d-c81f9237f8f6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6"
Jul 17 11:50:32.537915 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:32.537881    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:50:32.908636 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:32.908611    2566 scope.go:117] "RemoveContainer" containerID="d76b203ac449c0de68ede17e585ad9fc2b6a57594688f9d2a85401acd5e067ad"
Jul 17 11:50:32.909032 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:32.908863    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv_odh-ai-gateway-infra(da901a6e-8905-4050-9a5d-c81f9237f8f6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6"
Jul 17 11:50:36.003539 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.003507    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"]
Jul 17 11:50:36.008218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.008196    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"
Jul 17 11:50:36.015237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.015211    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"]
Jul 17 11:50:36.120842 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.120810    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7e389e-c555-48bf-ab34-efdd64220656-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-tsl96\" (UID: \"5b7e389e-c555-48bf-ab34-efdd64220656\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"
Jul 17 11:50:36.221678 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.221643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7e389e-c555-48bf-ab34-efdd64220656-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-tsl96\" (UID: \"5b7e389e-c555-48bf-ab34-efdd64220656\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"
Jul 17 11:50:36.222330 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.222301    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7e389e-c555-48bf-ab34-efdd64220656-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-tsl96\" (UID: \"5b7e389e-c555-48bf-ab34-efdd64220656\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"
Jul 17 11:50:36.320271 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.320180    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"
Jul 17 11:50:36.448470 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.448446    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"]
Jul 17 11:50:36.450381 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:50:36.450350    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7e389e_c555_48bf_ab34_efdd64220656.slice/crio-1e05d12e03e78a56e6f893f9ccb45e3dc054819f79c11b301a4db909f5258f82 WatchSource:0}: Error finding container 1e05d12e03e78a56e6f893f9ccb45e3dc054819f79c11b301a4db909f5258f82: Status 404 returned error can't find the container with id 1e05d12e03e78a56e6f893f9ccb45e3dc054819f79c11b301a4db909f5258f82
Jul 17 11:50:36.930648 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.930551    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" event={"ID":"5b7e389e-c555-48bf-ab34-efdd64220656","Type":"ContainerStarted","Data":"962d447aaac5639666912a5bdd990b50bee54cc9494b1d88ffc22f37cf70b511"}
Jul 17 11:50:36.930648 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.930645    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" event={"ID":"5b7e389e-c555-48bf-ab34-efdd64220656","Type":"ContainerStarted","Data":"1e05d12e03e78a56e6f893f9ccb45e3dc054819f79c11b301a4db909f5258f82"}
Jul 17 11:50:36.948228 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:36.948179    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" podStartSLOduration=1.948164927 podStartE2EDuration="1.948164927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:50:36.946843441 +0000 UTC m=+3219.803927386" watchObservedRunningTime="2026-07-17 11:50:36.948164927 +0000 UTC m=+3219.805248872"
Jul 17 11:50:37.937761 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:37.937726    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b7e389e-c555-48bf-ab34-efdd64220656" containerID="962d447aaac5639666912a5bdd990b50bee54cc9494b1d88ffc22f37cf70b511" exitCode=6
Jul 17 11:50:37.938256 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:37.937794    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" event={"ID":"5b7e389e-c555-48bf-ab34-efdd64220656","Type":"ContainerDied","Data":"962d447aaac5639666912a5bdd990b50bee54cc9494b1d88ffc22f37cf70b511"}
Jul 17 11:50:37.938256 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:37.938048    2566 scope.go:117] "RemoveContainer" containerID="962d447aaac5639666912a5bdd990b50bee54cc9494b1d88ffc22f37cf70b511"
Jul 17 11:50:38.944220 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:38.944175    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b7e389e-c555-48bf-ab34-efdd64220656" containerID="440e0124544146ce6821dbc47724055dd4f968882dae60576b21bdbaf5fe9f82" exitCode=6
Jul 17 11:50:38.944220 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:38.944218    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" event={"ID":"5b7e389e-c555-48bf-ab34-efdd64220656","Type":"ContainerDied","Data":"440e0124544146ce6821dbc47724055dd4f968882dae60576b21bdbaf5fe9f82"}
Jul 17 11:50:38.944744 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:38.944275    2566 scope.go:117] "RemoveContainer" containerID="962d447aaac5639666912a5bdd990b50bee54cc9494b1d88ffc22f37cf70b511"
Jul 17 11:50:38.944744 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:38.944495    2566 scope.go:117] "RemoveContainer" containerID="440e0124544146ce6821dbc47724055dd4f968882dae60576b21bdbaf5fe9f82"
Jul 17 11:50:38.944744 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:38.944720    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-tsl96_odh-ai-gateway-infra(5b7e389e-c555-48bf-ab34-efdd64220656)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" podUID="5b7e389e-c555-48bf-ab34-efdd64220656"
Jul 17 11:50:39.951468 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:39.951444    2566 scope.go:117] "RemoveContainer" containerID="440e0124544146ce6821dbc47724055dd4f968882dae60576b21bdbaf5fe9f82"
Jul 17 11:50:39.951937 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:39.951692    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-tsl96_odh-ai-gateway-infra(5b7e389e-c555-48bf-ab34-efdd64220656)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" podUID="5b7e389e-c555-48bf-ab34-efdd64220656"
Jul 17 11:50:47.654724 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:47.654681    2566 scope.go:117] "RemoveContainer" containerID="d76b203ac449c0de68ede17e585ad9fc2b6a57594688f9d2a85401acd5e067ad"
Jul 17 11:50:48.001458 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:48.001426    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" event={"ID":"da901a6e-8905-4050-9a5d-c81f9237f8f6","Type":"ContainerStarted","Data":"76bb2671ec37abb28f952edad29e650997d1ed70b1ac420b683cc0f4dc8ee1fe"}
Jul 17 11:50:48.021257 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:48.021202    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" podStartSLOduration=19.021187116 podStartE2EDuration="19.021187116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:50:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:50:48.020037909 +0000 UTC m=+3230.877121853" watchObservedRunningTime="2026-07-17 11:50:48.021187116 +0000 UTC m=+3230.878271097"
Jul 17 11:50:49.008720 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:49.008666    2566 generic.go:356] "Generic (PLEG): container finished" podID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerID="76bb2671ec37abb28f952edad29e650997d1ed70b1ac420b683cc0f4dc8ee1fe" exitCode=6
Jul 17 11:50:49.009119 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:49.008747    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" event={"ID":"da901a6e-8905-4050-9a5d-c81f9237f8f6","Type":"ContainerDied","Data":"76bb2671ec37abb28f952edad29e650997d1ed70b1ac420b683cc0f4dc8ee1fe"}
Jul 17 11:50:49.009119 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:49.008831    2566 scope.go:117] "RemoveContainer" containerID="d76b203ac449c0de68ede17e585ad9fc2b6a57594688f9d2a85401acd5e067ad"
Jul 17 11:50:49.009119 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:49.009093    2566 scope.go:117] "RemoveContainer" containerID="76bb2671ec37abb28f952edad29e650997d1ed70b1ac420b683cc0f4dc8ee1fe"
Jul 17 11:50:49.009377 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:49.009357    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv_odh-ai-gateway-infra(da901a6e-8905-4050-9a5d-c81f9237f8f6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6"
Jul 17 11:50:49.034202 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:49.034180    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"]
Jul 17 11:50:50.142460 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:50.142429    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"
Jul 17 11:50:50.227970 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:50.227936    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da901a6e-8905-4050-9a5d-c81f9237f8f6-maas-api-service-ca\") pod \"da901a6e-8905-4050-9a5d-c81f9237f8f6\" (UID: \"da901a6e-8905-4050-9a5d-c81f9237f8f6\") "
Jul 17 11:50:50.228304 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:50.228280    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da901a6e-8905-4050-9a5d-c81f9237f8f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da901a6e-8905-4050-9a5d-c81f9237f8f6" (UID: "da901a6e-8905-4050-9a5d-c81f9237f8f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:50:50.328546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:50.328512    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da901a6e-8905-4050-9a5d-c81f9237f8f6-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:50:51.022188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:51.022158    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv" event={"ID":"da901a6e-8905-4050-9a5d-c81f9237f8f6","Type":"ContainerDied","Data":"79240dbefa980b254cf84b32507a97e995cd830624f0665da8d4332c7595e61b"}
Jul 17 11:50:51.022188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:51.022178    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"
Jul 17 11:50:51.022441 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:51.022216    2566 scope.go:117] "RemoveContainer" containerID="76bb2671ec37abb28f952edad29e650997d1ed70b1ac420b683cc0f4dc8ee1fe"
Jul 17 11:50:51.045941 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:51.045915    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"]
Jul 17 11:50:51.051193 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:51.051171    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mm7pv"]
Jul 17 11:50:51.657190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:51.657161    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" path="/var/lib/kubelet/pods/da901a6e-8905-4050-9a5d-c81f9237f8f6/volumes"
Jul 17 11:50:52.652267 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:52.652231    2566 scope.go:117] "RemoveContainer" containerID="440e0124544146ce6821dbc47724055dd4f968882dae60576b21bdbaf5fe9f82"
Jul 17 11:50:53.035949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:53.035912    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" event={"ID":"5b7e389e-c555-48bf-ab34-efdd64220656","Type":"ContainerStarted","Data":"7aec6733d304438e9346293d308029abbc9c1cfc1b3133887a449c119f5ad4e5"}
Jul 17 11:50:54.042990 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:54.042955    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b7e389e-c555-48bf-ab34-efdd64220656" containerID="7aec6733d304438e9346293d308029abbc9c1cfc1b3133887a449c119f5ad4e5" exitCode=6
Jul 17 11:50:54.043446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:54.043028    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" event={"ID":"5b7e389e-c555-48bf-ab34-efdd64220656","Type":"ContainerDied","Data":"7aec6733d304438e9346293d308029abbc9c1cfc1b3133887a449c119f5ad4e5"}
Jul 17 11:50:54.043446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:54.043085    2566 scope.go:117] "RemoveContainer" containerID="440e0124544146ce6821dbc47724055dd4f968882dae60576b21bdbaf5fe9f82"
Jul 17 11:50:54.043446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:54.043329    2566 scope.go:117] "RemoveContainer" containerID="7aec6733d304438e9346293d308029abbc9c1cfc1b3133887a449c119f5ad4e5"
Jul 17 11:50:54.043594 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:50:54.043575    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-tsl96_odh-ai-gateway-infra(5b7e389e-c555-48bf-ab34-efdd64220656)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" podUID="5b7e389e-c555-48bf-ab34-efdd64220656"
Jul 17 11:50:54.071283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:54.071254    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"]
Jul 17 11:50:55.190562 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:55.190537    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"
Jul 17 11:50:55.266046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:55.266014    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7e389e-c555-48bf-ab34-efdd64220656-maas-api-service-ca\") pod \"5b7e389e-c555-48bf-ab34-efdd64220656\" (UID: \"5b7e389e-c555-48bf-ab34-efdd64220656\") "
Jul 17 11:50:55.266340 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:55.266320    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b7e389e-c555-48bf-ab34-efdd64220656-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5b7e389e-c555-48bf-ab34-efdd64220656" (UID: "5b7e389e-c555-48bf-ab34-efdd64220656"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:50:55.366563 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:55.366527    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7e389e-c555-48bf-ab34-efdd64220656-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:50:56.056849 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:56.056770    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"
Jul 17 11:50:56.057036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:56.056763    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96" event={"ID":"5b7e389e-c555-48bf-ab34-efdd64220656","Type":"ContainerDied","Data":"1e05d12e03e78a56e6f893f9ccb45e3dc054819f79c11b301a4db909f5258f82"}
Jul 17 11:50:56.057036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:56.056915    2566 scope.go:117] "RemoveContainer" containerID="7aec6733d304438e9346293d308029abbc9c1cfc1b3133887a449c119f5ad4e5"
Jul 17 11:50:56.077538 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:56.077514    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"]
Jul 17 11:50:56.080727 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:56.080696    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-tsl96"]
Jul 17 11:50:57.657466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:57.657435    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" path="/var/lib/kubelet/pods/5b7e389e-c555-48bf-ab34-efdd64220656/volumes"
Jul 17 11:50:59.314545 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.314510    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"]
Jul 17 11:50:59.315096 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315075    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerName="revoke-keys"
Jul 17 11:50:59.315181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315099    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerName="revoke-keys"
Jul 17 11:50:59.315181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315115    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerName="revoke-keys"
Jul 17 11:50:59.315181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315122    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerName="revoke-keys"
Jul 17 11:50:59.315181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315133    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" containerName="revoke-keys"
Jul 17 11:50:59.315181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315143    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" containerName="revoke-keys"
Jul 17 11:50:59.315181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315157    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerName="revoke-keys"
Jul 17 11:50:59.315181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315165    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerName="revoke-keys"
Jul 17 11:50:59.315181 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315177    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" containerName="revoke-keys"
Jul 17 11:50:59.315553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315185    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" containerName="revoke-keys"
Jul 17 11:50:59.315553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315294    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" containerName="revoke-keys"
Jul 17 11:50:59.315553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315309    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerName="revoke-keys"
Jul 17 11:50:59.315553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315320    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerName="revoke-keys"
Jul 17 11:50:59.315553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315330    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da901a6e-8905-4050-9a5d-c81f9237f8f6" containerName="revoke-keys"
Jul 17 11:50:59.315553 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.315340    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" containerName="revoke-keys"
Jul 17 11:50:59.319947 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.319926    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"
Jul 17 11:50:59.323171 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.323144    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:50:59.324040 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.324018    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:50:59.325735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.325699    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"]
Jul 17 11:50:59.401459 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.401429    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc5b8068-1526-4ecc-8864-71acaf55b3ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-68c72\" (UID: \"fc5b8068-1526-4ecc-8864-71acaf55b3ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"
Jul 17 11:50:59.501886 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.501862    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc5b8068-1526-4ecc-8864-71acaf55b3ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-68c72\" (UID: \"fc5b8068-1526-4ecc-8864-71acaf55b3ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"
Jul 17 11:50:59.502453 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.502430    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc5b8068-1526-4ecc-8864-71acaf55b3ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-68c72\" (UID: \"fc5b8068-1526-4ecc-8864-71acaf55b3ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"
Jul 17 11:50:59.632056 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.632031    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"
Jul 17 11:50:59.748352 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.748323    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:50:59.761841 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:50:59.761822    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"]
Jul 17 11:50:59.763999 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:50:59.763972    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5b8068_1526_4ecc_8864_71acaf55b3ad.slice/crio-31fa4aef15cadc0bccb9f99b4fc5589d290c68964b2dca350a70e9efd2364b73 WatchSource:0}: Error finding container 31fa4aef15cadc0bccb9f99b4fc5589d290c68964b2dca350a70e9efd2364b73: Status 404 returned error can't find the container with id 31fa4aef15cadc0bccb9f99b4fc5589d290c68964b2dca350a70e9efd2364b73
Jul 17 11:51:00.079150 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:00.079120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" event={"ID":"fc5b8068-1526-4ecc-8864-71acaf55b3ad","Type":"ContainerStarted","Data":"4619f11325083a880f5613d98b292f86055e5adb7e90585ec055376c2d795856"}
Jul 17 11:51:00.079304 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:00.079185    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" event={"ID":"fc5b8068-1526-4ecc-8864-71acaf55b3ad","Type":"ContainerStarted","Data":"31fa4aef15cadc0bccb9f99b4fc5589d290c68964b2dca350a70e9efd2364b73"}
Jul 17 11:51:00.096979 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:00.096933    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" podStartSLOduration=1.096918372 podStartE2EDuration="1.096918372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:50:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:51:00.095980446 +0000 UTC m=+3242.953064386" watchObservedRunningTime="2026-07-17 11:51:00.096918372 +0000 UTC m=+3242.954002316"
Jul 17 11:51:01.086716 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:01.086666    2566 generic.go:356] "Generic (PLEG): container finished" podID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerID="4619f11325083a880f5613d98b292f86055e5adb7e90585ec055376c2d795856" exitCode=6
Jul 17 11:51:01.087115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:01.086748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" event={"ID":"fc5b8068-1526-4ecc-8864-71acaf55b3ad","Type":"ContainerDied","Data":"4619f11325083a880f5613d98b292f86055e5adb7e90585ec055376c2d795856"}
Jul 17 11:51:01.087115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:01.086925    2566 scope.go:117] "RemoveContainer" containerID="4619f11325083a880f5613d98b292f86055e5adb7e90585ec055376c2d795856"
Jul 17 11:51:02.094783 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:02.094750    2566 generic.go:356] "Generic (PLEG): container finished" podID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerID="1e1bf752e8a5aed10950038d3e23ecd2978d140df4eb4d489ca256fd0f111c76" exitCode=6
Jul 17 11:51:02.095215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:02.094787    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" event={"ID":"fc5b8068-1526-4ecc-8864-71acaf55b3ad","Type":"ContainerDied","Data":"1e1bf752e8a5aed10950038d3e23ecd2978d140df4eb4d489ca256fd0f111c76"}
Jul 17 11:51:02.095215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:02.094852    2566 scope.go:117] "RemoveContainer" containerID="4619f11325083a880f5613d98b292f86055e5adb7e90585ec055376c2d795856"
Jul 17 11:51:02.095215 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:02.095042    2566 scope.go:117] "RemoveContainer" containerID="1e1bf752e8a5aed10950038d3e23ecd2978d140df4eb4d489ca256fd0f111c76"
Jul 17 11:51:02.095351 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:02.095244    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-68c72_odh-ai-gateway-infra(fc5b8068-1526-4ecc-8864-71acaf55b3ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad"
Jul 17 11:51:03.102409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:03.102377    2566 scope.go:117] "RemoveContainer" containerID="1e1bf752e8a5aed10950038d3e23ecd2978d140df4eb4d489ca256fd0f111c76"
Jul 17 11:51:03.102792 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:03.102616    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-68c72_odh-ai-gateway-infra(fc5b8068-1526-4ecc-8864-71acaf55b3ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad"
Jul 17 11:51:14.946586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:14.946549    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:51:15.652019 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:15.651986    2566 scope.go:117] "RemoveContainer" containerID="1e1bf752e8a5aed10950038d3e23ecd2978d140df4eb4d489ca256fd0f111c76"
Jul 17 11:51:16.175806 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:16.175779    2566 generic.go:356] "Generic (PLEG): container finished" podID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerID="2d5db96a308f7fe8ce81c97ba1e32381c7333c97732b231bb3cd797e0cc3fc4e" exitCode=6
Jul 17 11:51:16.176155 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:16.175847    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" event={"ID":"fc5b8068-1526-4ecc-8864-71acaf55b3ad","Type":"ContainerDied","Data":"2d5db96a308f7fe8ce81c97ba1e32381c7333c97732b231bb3cd797e0cc3fc4e"}
Jul 17 11:51:16.176155 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:16.175904    2566 scope.go:117] "RemoveContainer" containerID="1e1bf752e8a5aed10950038d3e23ecd2978d140df4eb4d489ca256fd0f111c76"
Jul 17 11:51:16.176267 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:16.176249    2566 scope.go:117] "RemoveContainer" containerID="2d5db96a308f7fe8ce81c97ba1e32381c7333c97732b231bb3cd797e0cc3fc4e"
Jul 17 11:51:16.176476 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:16.176454    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-68c72_odh-ai-gateway-infra(fc5b8068-1526-4ecc-8864-71acaf55b3ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad"
Jul 17 11:51:17.203334 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:17.203302    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"]
Jul 17 11:51:17.338312 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:17.338290    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"
Jul 17 11:51:17.461661 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:17.461581    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc5b8068-1526-4ecc-8864-71acaf55b3ad-maas-api-service-ca\") pod \"fc5b8068-1526-4ecc-8864-71acaf55b3ad\" (UID: \"fc5b8068-1526-4ecc-8864-71acaf55b3ad\") "
Jul 17 11:51:17.461949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:17.461929    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc5b8068-1526-4ecc-8864-71acaf55b3ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc5b8068-1526-4ecc-8864-71acaf55b3ad" (UID: "fc5b8068-1526-4ecc-8864-71acaf55b3ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:51:17.562196 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:17.562163    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc5b8068-1526-4ecc-8864-71acaf55b3ad-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:51:18.189995 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:18.189955    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"
Jul 17 11:51:18.189995 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:18.189975    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72" event={"ID":"fc5b8068-1526-4ecc-8864-71acaf55b3ad","Type":"ContainerDied","Data":"31fa4aef15cadc0bccb9f99b4fc5589d290c68964b2dca350a70e9efd2364b73"}
Jul 17 11:51:18.190253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:18.190037    2566 scope.go:117] "RemoveContainer" containerID="2d5db96a308f7fe8ce81c97ba1e32381c7333c97732b231bb3cd797e0cc3fc4e"
Jul 17 11:51:18.211977 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:18.211952    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"]
Jul 17 11:51:18.218309 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:18.218285    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-68c72"]
Jul 17 11:51:19.657576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:19.657547    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" path="/var/lib/kubelet/pods/fc5b8068-1526-4ecc-8864-71acaf55b3ad/volumes"
Jul 17 11:51:28.264594 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.264553    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"]
Jul 17 11:51:28.265114 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.265091    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" containerName="revoke-keys"
Jul 17 11:51:28.265194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.265118    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" containerName="revoke-keys"
Jul 17 11:51:28.265194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.265129    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerName="revoke-keys"
Jul 17 11:51:28.265194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.265137    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerName="revoke-keys"
Jul 17 11:51:28.265194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.265157    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerName="revoke-keys"
Jul 17 11:51:28.265194 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.265166    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerName="revoke-keys"
Jul 17 11:51:28.265462 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.265283    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerName="revoke-keys"
Jul 17 11:51:28.265462 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.265297    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerName="revoke-keys"
Jul 17 11:51:28.265462 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.265308    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7e389e-c555-48bf-ab34-efdd64220656" containerName="revoke-keys"
Jul 17 11:51:28.269745 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.269724    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"
Jul 17 11:51:28.273940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.273921    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:51:28.274262 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.274244    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:51:28.280377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.280359    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"]
Jul 17 11:51:28.362057 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.362032    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e621f653-be26-4a66-b9b1-df8d88e380ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-8lld6\" (UID: \"e621f653-be26-4a66-b9b1-df8d88e380ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"
Jul 17 11:51:28.462701 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.462678    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e621f653-be26-4a66-b9b1-df8d88e380ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-8lld6\" (UID: \"e621f653-be26-4a66-b9b1-df8d88e380ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"
Jul 17 11:51:28.463274 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.463252    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e621f653-be26-4a66-b9b1-df8d88e380ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-8lld6\" (UID: \"e621f653-be26-4a66-b9b1-df8d88e380ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"
Jul 17 11:51:28.581023 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.581000    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"
Jul 17 11:51:28.712670 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:28.712643    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"]
Jul 17 11:51:28.714080 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:51:28.714051    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode621f653_be26_4a66_b9b1_df8d88e380ad.slice/crio-105b76ba655119e8fb78a809a88dddac7e1ea01360b4da1a23b90a293ad2207b WatchSource:0}: Error finding container 105b76ba655119e8fb78a809a88dddac7e1ea01360b4da1a23b90a293ad2207b: Status 404 returned error can't find the container with id 105b76ba655119e8fb78a809a88dddac7e1ea01360b4da1a23b90a293ad2207b
Jul 17 11:51:29.220377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.220295    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"]
Jul 17 11:51:29.220773 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.220756    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerName="revoke-keys"
Jul 17 11:51:29.220868 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.220775    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerName="revoke-keys"
Jul 17 11:51:29.220921 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.220874    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc5b8068-1526-4ecc-8864-71acaf55b3ad" containerName="revoke-keys"
Jul 17 11:51:29.223901 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.223881    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"
Jul 17 11:51:29.233098 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.233070    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"]
Jul 17 11:51:29.250407 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.250378    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" event={"ID":"e621f653-be26-4a66-b9b1-df8d88e380ad","Type":"ContainerStarted","Data":"7fc839648d69a3857bf0fce8ade86bcfb5e9062973189af1d301042ad9a7a82d"}
Jul 17 11:51:29.250519 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.250422    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" event={"ID":"e621f653-be26-4a66-b9b1-df8d88e380ad","Type":"ContainerStarted","Data":"105b76ba655119e8fb78a809a88dddac7e1ea01360b4da1a23b90a293ad2207b"}
Jul 17 11:51:29.270136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.270092    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" podStartSLOduration=1.270079797 podStartE2EDuration="1.270079797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:51:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:51:29.26878096 +0000 UTC m=+3272.125864918" watchObservedRunningTime="2026-07-17 11:51:29.270079797 +0000 UTC m=+3272.127163760"
Jul 17 11:51:29.369406 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.369384    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d036b439-faad-43f1-917e-665ebef960c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk\" (UID: \"d036b439-faad-43f1-917e-665ebef960c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"
Jul 17 11:51:29.469991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.469964    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d036b439-faad-43f1-917e-665ebef960c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk\" (UID: \"d036b439-faad-43f1-917e-665ebef960c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"
Jul 17 11:51:29.470550 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.470508    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d036b439-faad-43f1-917e-665ebef960c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk\" (UID: \"d036b439-faad-43f1-917e-665ebef960c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"
Jul 17 11:51:29.535303 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.535282    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"
Jul 17 11:51:29.871797 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:29.871741    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"]
Jul 17 11:51:29.874429 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:51:29.874405    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd036b439_faad_43f1_917e_665ebef960c3.slice/crio-51513ca3a3d188db34c7edd2a0b460f52b5811a232ec64837f17f0ec4c2c4796 WatchSource:0}: Error finding container 51513ca3a3d188db34c7edd2a0b460f52b5811a232ec64837f17f0ec4c2c4796: Status 404 returned error can't find the container with id 51513ca3a3d188db34c7edd2a0b460f52b5811a232ec64837f17f0ec4c2c4796
Jul 17 11:51:30.257627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:30.257589    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" event={"ID":"d036b439-faad-43f1-917e-665ebef960c3","Type":"ContainerStarted","Data":"53d9d6cc2dc72b0cf8d47b50f8e1b11082802a157e9b9fd0c67c8cf41a6b9c3a"}
Jul 17 11:51:30.257830 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:30.257652    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" event={"ID":"d036b439-faad-43f1-917e-665ebef960c3","Type":"ContainerStarted","Data":"51513ca3a3d188db34c7edd2a0b460f52b5811a232ec64837f17f0ec4c2c4796"}
Jul 17 11:51:30.259143 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:30.259117    2566 generic.go:356] "Generic (PLEG): container finished" podID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerID="7fc839648d69a3857bf0fce8ade86bcfb5e9062973189af1d301042ad9a7a82d" exitCode=6
Jul 17 11:51:30.259266 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:30.259199    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" event={"ID":"e621f653-be26-4a66-b9b1-df8d88e380ad","Type":"ContainerDied","Data":"7fc839648d69a3857bf0fce8ade86bcfb5e9062973189af1d301042ad9a7a82d"}
Jul 17 11:51:30.259313 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:30.259293    2566 scope.go:117] "RemoveContainer" containerID="7fc839648d69a3857bf0fce8ade86bcfb5e9062973189af1d301042ad9a7a82d"
Jul 17 11:51:30.275715 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:30.275665    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" podStartSLOduration=1.2756516150000001 podStartE2EDuration="1.275651615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:51:30.274356339 +0000 UTC m=+3273.131440283" watchObservedRunningTime="2026-07-17 11:51:30.275651615 +0000 UTC m=+3273.132735559"
Jul 17 11:51:31.266153 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:31.266117    2566 generic.go:356] "Generic (PLEG): container finished" podID="d036b439-faad-43f1-917e-665ebef960c3" containerID="53d9d6cc2dc72b0cf8d47b50f8e1b11082802a157e9b9fd0c67c8cf41a6b9c3a" exitCode=6
Jul 17 11:51:31.266373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:31.266208    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" event={"ID":"d036b439-faad-43f1-917e-665ebef960c3","Type":"ContainerDied","Data":"53d9d6cc2dc72b0cf8d47b50f8e1b11082802a157e9b9fd0c67c8cf41a6b9c3a"}
Jul 17 11:51:31.266373 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:31.266363    2566 scope.go:117] "RemoveContainer" containerID="53d9d6cc2dc72b0cf8d47b50f8e1b11082802a157e9b9fd0c67c8cf41a6b9c3a"
Jul 17 11:51:31.268035 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:31.268013    2566 generic.go:356] "Generic (PLEG): container finished" podID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerID="71b8a724a41b59ddaf74289039c49dc5df86663b937c46d57ec77a81319fae76" exitCode=6
Jul 17 11:51:31.268165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:31.268096    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" event={"ID":"e621f653-be26-4a66-b9b1-df8d88e380ad","Type":"ContainerDied","Data":"71b8a724a41b59ddaf74289039c49dc5df86663b937c46d57ec77a81319fae76"}
Jul 17 11:51:31.268318 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:31.268167    2566 scope.go:117] "RemoveContainer" containerID="7fc839648d69a3857bf0fce8ade86bcfb5e9062973189af1d301042ad9a7a82d"
Jul 17 11:51:31.268318 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:31.268244    2566 scope.go:117] "RemoveContainer" containerID="71b8a724a41b59ddaf74289039c49dc5df86663b937c46d57ec77a81319fae76"
Jul 17 11:51:31.268430 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:31.268408    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-8lld6_odh-ai-gateway-infra(e621f653-be26-4a66-b9b1-df8d88e380ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad"
Jul 17 11:51:32.275424 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:32.275341    2566 generic.go:356] "Generic (PLEG): container finished" podID="d036b439-faad-43f1-917e-665ebef960c3" containerID="7b6805301fec7ab1de5135c07154c238d594580af44cbb76bed0efc04a34a27f" exitCode=6
Jul 17 11:51:32.275883 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:32.275426    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" event={"ID":"d036b439-faad-43f1-917e-665ebef960c3","Type":"ContainerDied","Data":"7b6805301fec7ab1de5135c07154c238d594580af44cbb76bed0efc04a34a27f"}
Jul 17 11:51:32.275883 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:32.275489    2566 scope.go:117] "RemoveContainer" containerID="53d9d6cc2dc72b0cf8d47b50f8e1b11082802a157e9b9fd0c67c8cf41a6b9c3a"
Jul 17 11:51:32.275883 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:32.275592    2566 scope.go:117] "RemoveContainer" containerID="7b6805301fec7ab1de5135c07154c238d594580af44cbb76bed0efc04a34a27f"
Jul 17 11:51:32.275883 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:32.275869    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk_odh-ai-gateway-infra(d036b439-faad-43f1-917e-665ebef960c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" podUID="d036b439-faad-43f1-917e-665ebef960c3"
Jul 17 11:51:32.277275 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:32.277260    2566 scope.go:117] "RemoveContainer" containerID="71b8a724a41b59ddaf74289039c49dc5df86663b937c46d57ec77a81319fae76"
Jul 17 11:51:32.277434 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:32.277416    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-8lld6_odh-ai-gateway-infra(e621f653-be26-4a66-b9b1-df8d88e380ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad"
Jul 17 11:51:33.284046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:33.284018    2566 scope.go:117] "RemoveContainer" containerID="7b6805301fec7ab1de5135c07154c238d594580af44cbb76bed0efc04a34a27f"
Jul 17 11:51:33.284451 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:33.284218    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk_odh-ai-gateway-infra(d036b439-faad-43f1-917e-665ebef960c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" podUID="d036b439-faad-43f1-917e-665ebef960c3"
Jul 17 11:51:35.837493 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:35.837463    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:51:40.511494 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:40.511456    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"]
Jul 17 11:51:46.651660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:46.651625    2566 scope.go:117] "RemoveContainer" containerID="7b6805301fec7ab1de5135c07154c238d594580af44cbb76bed0efc04a34a27f"
Jul 17 11:51:46.651660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:46.651659    2566 scope.go:117] "RemoveContainer" containerID="71b8a724a41b59ddaf74289039c49dc5df86663b937c46d57ec77a81319fae76"
Jul 17 11:51:47.357394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:47.357370    2566 generic.go:356] "Generic (PLEG): container finished" podID="d036b439-faad-43f1-917e-665ebef960c3" containerID="239d556b7a04a276559e4704b4f54e90de1ab05cb5046fe2d3e8a9bc5b31f50c" exitCode=6
Jul 17 11:51:47.357524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:47.357444    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" event={"ID":"d036b439-faad-43f1-917e-665ebef960c3","Type":"ContainerDied","Data":"239d556b7a04a276559e4704b4f54e90de1ab05cb5046fe2d3e8a9bc5b31f50c"}
Jul 17 11:51:47.357524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:47.357496    2566 scope.go:117] "RemoveContainer" containerID="7b6805301fec7ab1de5135c07154c238d594580af44cbb76bed0efc04a34a27f"
Jul 17 11:51:47.357848 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:47.357824    2566 scope.go:117] "RemoveContainer" containerID="239d556b7a04a276559e4704b4f54e90de1ab05cb5046fe2d3e8a9bc5b31f50c"
Jul 17 11:51:47.358080 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:47.358055    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk_odh-ai-gateway-infra(d036b439-faad-43f1-917e-665ebef960c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" podUID="d036b439-faad-43f1-917e-665ebef960c3"
Jul 17 11:51:47.359326 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:47.359309    2566 generic.go:356] "Generic (PLEG): container finished" podID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerID="324f6e6213735802d83d4829a5053ae7acfaa66611c4a40715f96c7c8106354d" exitCode=6
Jul 17 11:51:47.359419 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:47.359343    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" event={"ID":"e621f653-be26-4a66-b9b1-df8d88e380ad","Type":"ContainerDied","Data":"324f6e6213735802d83d4829a5053ae7acfaa66611c4a40715f96c7c8106354d"}
Jul 17 11:51:47.359620 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:47.359607    2566 scope.go:117] "RemoveContainer" containerID="324f6e6213735802d83d4829a5053ae7acfaa66611c4a40715f96c7c8106354d"
Jul 17 11:51:47.359807 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:47.359788    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-8lld6_odh-ai-gateway-infra(e621f653-be26-4a66-b9b1-df8d88e380ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad"
Jul 17 11:51:47.385717 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:47.385689    2566 scope.go:117] "RemoveContainer" containerID="71b8a724a41b59ddaf74289039c49dc5df86663b937c46d57ec77a81319fae76"
Jul 17 11:51:48.384874 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.384841    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"]
Jul 17 11:51:48.403173 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.403134    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"]
Jul 17 11:51:48.531884 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.531860    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"
Jul 17 11:51:48.538771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.538750    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d036b439-faad-43f1-917e-665ebef960c3-maas-api-service-ca\") pod \"d036b439-faad-43f1-917e-665ebef960c3\" (UID: \"d036b439-faad-43f1-917e-665ebef960c3\") "
Jul 17 11:51:48.539120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.539101    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d036b439-faad-43f1-917e-665ebef960c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d036b439-faad-43f1-917e-665ebef960c3" (UID: "d036b439-faad-43f1-917e-665ebef960c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:51:48.563852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.563829    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"
Jul 17 11:51:48.640288 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.640210    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e621f653-be26-4a66-b9b1-df8d88e380ad-maas-api-service-ca\") pod \"e621f653-be26-4a66-b9b1-df8d88e380ad\" (UID: \"e621f653-be26-4a66-b9b1-df8d88e380ad\") "
Jul 17 11:51:48.640437 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.640424    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d036b439-faad-43f1-917e-665ebef960c3-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:51:48.640567 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.640547    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e621f653-be26-4a66-b9b1-df8d88e380ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e621f653-be26-4a66-b9b1-df8d88e380ad" (UID: "e621f653-be26-4a66-b9b1-df8d88e380ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:51:48.741725 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:48.741670    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e621f653-be26-4a66-b9b1-df8d88e380ad-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:51:49.362556 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.362509    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"]
Jul 17 11:51:49.363403 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363371    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerName="revoke-keys"
Jul 17 11:51:49.363403 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363399    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerName="revoke-keys"
Jul 17 11:51:49.363611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363415    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerName="revoke-keys"
Jul 17 11:51:49.363611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363425    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerName="revoke-keys"
Jul 17 11:51:49.363611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363447    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d036b439-faad-43f1-917e-665ebef960c3" containerName="revoke-keys"
Jul 17 11:51:49.363611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363456    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d036b439-faad-43f1-917e-665ebef960c3" containerName="revoke-keys"
Jul 17 11:51:49.363611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363480    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d036b439-faad-43f1-917e-665ebef960c3" containerName="revoke-keys"
Jul 17 11:51:49.363611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363490    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d036b439-faad-43f1-917e-665ebef960c3" containerName="revoke-keys"
Jul 17 11:51:49.363611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363530    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d036b439-faad-43f1-917e-665ebef960c3" containerName="revoke-keys"
Jul 17 11:51:49.363611 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363539    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d036b439-faad-43f1-917e-665ebef960c3" containerName="revoke-keys"
Jul 17 11:51:49.364027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363727    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerName="revoke-keys"
Jul 17 11:51:49.364027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363743    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerName="revoke-keys"
Jul 17 11:51:49.364027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363752    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d036b439-faad-43f1-917e-665ebef960c3" containerName="revoke-keys"
Jul 17 11:51:49.364027 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.363774    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d036b439-faad-43f1-917e-665ebef960c3" containerName="revoke-keys"
Jul 17 11:51:49.367468 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.367450    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"
Jul 17 11:51:49.372209 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.372181    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"]
Jul 17 11:51:49.376358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.376313    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"
Jul 17 11:51:49.376358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.376323    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk" event={"ID":"d036b439-faad-43f1-917e-665ebef960c3","Type":"ContainerDied","Data":"51513ca3a3d188db34c7edd2a0b460f52b5811a232ec64837f17f0ec4c2c4796"}
Jul 17 11:51:49.376496 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.376412    2566 scope.go:117] "RemoveContainer" containerID="239d556b7a04a276559e4704b4f54e90de1ab05cb5046fe2d3e8a9bc5b31f50c"
Jul 17 11:51:49.380014 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.378162    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6" event={"ID":"e621f653-be26-4a66-b9b1-df8d88e380ad","Type":"ContainerDied","Data":"105b76ba655119e8fb78a809a88dddac7e1ea01360b4da1a23b90a293ad2207b"}
Jul 17 11:51:49.380014 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.378252    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"
Jul 17 11:51:49.388186 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.388022    2566 scope.go:117] "RemoveContainer" containerID="324f6e6213735802d83d4829a5053ae7acfaa66611c4a40715f96c7c8106354d"
Jul 17 11:51:49.404758 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.404738    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"]
Jul 17 11:51:49.411802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.411784    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8lld6"]
Jul 17 11:51:49.429379 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.429354    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"]
Jul 17 11:51:49.431216 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.431197    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-nbndk"]
Jul 17 11:51:49.448245 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.448225    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0831eed5-422c-4eff-862b-3b2bab15ea12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h\" (UID: \"0831eed5-422c-4eff-862b-3b2bab15ea12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"
Jul 17 11:51:49.548981 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.548957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0831eed5-422c-4eff-862b-3b2bab15ea12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h\" (UID: \"0831eed5-422c-4eff-862b-3b2bab15ea12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"
Jul 17 11:51:49.549530 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.549513    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0831eed5-422c-4eff-862b-3b2bab15ea12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h\" (UID: \"0831eed5-422c-4eff-862b-3b2bab15ea12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"
Jul 17 11:51:49.657465 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.657404    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d036b439-faad-43f1-917e-665ebef960c3" path="/var/lib/kubelet/pods/d036b439-faad-43f1-917e-665ebef960c3/volumes"
Jul 17 11:51:49.657772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.657758    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" path="/var/lib/kubelet/pods/e621f653-be26-4a66-b9b1-df8d88e380ad/volumes"
Jul 17 11:51:49.681768 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:49.681735    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"
Jul 17 11:51:50.019530 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:50.019460    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"]
Jul 17 11:51:50.022441 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:51:50.022412    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0831eed5_422c_4eff_862b_3b2bab15ea12.slice/crio-46c033ada66a44b09cfe1bce8a2f70f4926323eb783fd3019822f02838ce73b1 WatchSource:0}: Error finding container 46c033ada66a44b09cfe1bce8a2f70f4926323eb783fd3019822f02838ce73b1: Status 404 returned error can't find the container with id 46c033ada66a44b09cfe1bce8a2f70f4926323eb783fd3019822f02838ce73b1
Jul 17 11:51:50.388625 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:50.388583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" event={"ID":"0831eed5-422c-4eff-862b-3b2bab15ea12","Type":"ContainerStarted","Data":"49cf0b46dc6908db09c53464bccfa120233befef4b796b7b87ddcca97e540f54"}
Jul 17 11:51:50.388999 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:50.388634    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" event={"ID":"0831eed5-422c-4eff-862b-3b2bab15ea12","Type":"ContainerStarted","Data":"46c033ada66a44b09cfe1bce8a2f70f4926323eb783fd3019822f02838ce73b1"}
Jul 17 11:51:50.405498 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:50.405452    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" podStartSLOduration=1.405438215 podStartE2EDuration="1.405438215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:51:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:51:50.404315355 +0000 UTC m=+3293.261399301" watchObservedRunningTime="2026-07-17 11:51:50.405438215 +0000 UTC m=+3293.262522160"
Jul 17 11:51:51.396409 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:51.396377    2566 generic.go:356] "Generic (PLEG): container finished" podID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerID="49cf0b46dc6908db09c53464bccfa120233befef4b796b7b87ddcca97e540f54" exitCode=6
Jul 17 11:51:51.396844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:51.396461    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" event={"ID":"0831eed5-422c-4eff-862b-3b2bab15ea12","Type":"ContainerDied","Data":"49cf0b46dc6908db09c53464bccfa120233befef4b796b7b87ddcca97e540f54"}
Jul 17 11:51:51.396844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:51.396593    2566 scope.go:117] "RemoveContainer" containerID="49cf0b46dc6908db09c53464bccfa120233befef4b796b7b87ddcca97e540f54"
Jul 17 11:51:52.403807 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:52.403732    2566 generic.go:356] "Generic (PLEG): container finished" podID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerID="1ee88406ca8e52a8de7d0e265ad8eaa94ca732197077db2437929c026316f34e" exitCode=6
Jul 17 11:51:52.404212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:52.403823    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" event={"ID":"0831eed5-422c-4eff-862b-3b2bab15ea12","Type":"ContainerDied","Data":"1ee88406ca8e52a8de7d0e265ad8eaa94ca732197077db2437929c026316f34e"}
Jul 17 11:51:52.404212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:52.403883    2566 scope.go:117] "RemoveContainer" containerID="49cf0b46dc6908db09c53464bccfa120233befef4b796b7b87ddcca97e540f54"
Jul 17 11:51:52.404212 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:52.404112    2566 scope.go:117] "RemoveContainer" containerID="1ee88406ca8e52a8de7d0e265ad8eaa94ca732197077db2437929c026316f34e"
Jul 17 11:51:52.404337 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:52.404314    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h_odh-ai-gateway-infra(0831eed5-422c-4eff-862b-3b2bab15ea12)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12"
Jul 17 11:51:53.411071 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:53.411047    2566 scope.go:117] "RemoveContainer" containerID="1ee88406ca8e52a8de7d0e265ad8eaa94ca732197077db2437929c026316f34e"
Jul 17 11:51:53.411418 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:53.411240    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h_odh-ai-gateway-infra(0831eed5-422c-4eff-862b-3b2bab15ea12)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12"
Jul 17 11:51:54.543203 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:54.543168    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:51:55.531779 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.531735    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j" podUID="fc5b3650-e267-438d-8a01-76dd7752723a" containerName="main" containerID="cri-o://34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc" gracePeriod=45
Jul 17 11:51:55.790776 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.790718    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:51:55.904949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.904912    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-tmp-dir\") pod \"fc5b3650-e267-438d-8a01-76dd7752723a\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") "
Jul 17 11:51:55.904949 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.904952    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-dshm\") pod \"fc5b3650-e267-438d-8a01-76dd7752723a\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") "
Jul 17 11:51:55.905209 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.904994    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-home\") pod \"fc5b3650-e267-438d-8a01-76dd7752723a\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") "
Jul 17 11:51:55.905209 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.905014    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqx77\" (UniqueName: \"kubernetes.io/projected/fc5b3650-e267-438d-8a01-76dd7752723a-kube-api-access-lqx77\") pod \"fc5b3650-e267-438d-8a01-76dd7752723a\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") "
Jul 17 11:51:55.905209 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.905029    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-model-cache\") pod \"fc5b3650-e267-438d-8a01-76dd7752723a\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") "
Jul 17 11:51:55.905209 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.905054    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc5b3650-e267-438d-8a01-76dd7752723a-tls-certs\") pod \"fc5b3650-e267-438d-8a01-76dd7752723a\" (UID: \"fc5b3650-e267-438d-8a01-76dd7752723a\") "
Jul 17 11:51:55.905415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.905265    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fc5b3650-e267-438d-8a01-76dd7752723a" (UID: "fc5b3650-e267-438d-8a01-76dd7752723a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:51:55.905415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.905285    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-home" (OuterVolumeSpecName: "home") pod "fc5b3650-e267-438d-8a01-76dd7752723a" (UID: "fc5b3650-e267-438d-8a01-76dd7752723a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:51:55.905415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.905355    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-model-cache" (OuterVolumeSpecName: "model-cache") pod "fc5b3650-e267-438d-8a01-76dd7752723a" (UID: "fc5b3650-e267-438d-8a01-76dd7752723a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:51:55.905526 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.905475    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:51:55.905526 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.905498    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:51:55.905526 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.905512    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:51:55.907312 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.907289    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-dshm" (OuterVolumeSpecName: "dshm") pod "fc5b3650-e267-438d-8a01-76dd7752723a" (UID: "fc5b3650-e267-438d-8a01-76dd7752723a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:51:55.907423 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.907335    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc5b3650-e267-438d-8a01-76dd7752723a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fc5b3650-e267-438d-8a01-76dd7752723a" (UID: "fc5b3650-e267-438d-8a01-76dd7752723a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 11:51:55.907423 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:55.907341    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc5b3650-e267-438d-8a01-76dd7752723a-kube-api-access-lqx77" (OuterVolumeSpecName: "kube-api-access-lqx77") pod "fc5b3650-e267-438d-8a01-76dd7752723a" (UID: "fc5b3650-e267-438d-8a01-76dd7752723a"). InnerVolumeSpecName "kube-api-access-lqx77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:51:56.006286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.006251    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc5b3650-e267-438d-8a01-76dd7752723a-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:51:56.006286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.006280    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqx77\" (UniqueName: \"kubernetes.io/projected/fc5b3650-e267-438d-8a01-76dd7752723a-kube-api-access-lqx77\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:51:56.006286 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.006293    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc5b3650-e267-438d-8a01-76dd7752723a-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:51:56.428124 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.428091    2566 generic.go:356] "Generic (PLEG): container finished" podID="fc5b3650-e267-438d-8a01-76dd7752723a" containerID="34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc" exitCode=0
Jul 17 11:51:56.428328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.428135    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"
Jul 17 11:51:56.428328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.428179    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j" event={"ID":"fc5b3650-e267-438d-8a01-76dd7752723a","Type":"ContainerDied","Data":"34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc"}
Jul 17 11:51:56.428328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.428224    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j" event={"ID":"fc5b3650-e267-438d-8a01-76dd7752723a","Type":"ContainerDied","Data":"8e57b235898d3dd8256db9a5fb18bb7e8a6ed982923f74d327728a2f4e124d15"}
Jul 17 11:51:56.428328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.428246    2566 scope.go:117] "RemoveContainer" containerID="34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc"
Jul 17 11:51:56.437774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.437755    2566 scope.go:117] "RemoveContainer" containerID="34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc"
Jul 17 11:51:56.438045 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:51:56.438026    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc\": container with ID starting with 34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc not found: ID does not exist" containerID="34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc"
Jul 17 11:51:56.438115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.438052    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc"} err="failed to get container status \"34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc\": rpc error: code = NotFound desc = could not find container \"34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc\": container with ID starting with 34ce5a9d1ce67e22295a488d0a0f175d2da981824d0e266a1437496a3f99a5dc not found: ID does not exist"
Jul 17 11:51:56.451792 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.451771    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"]
Jul 17 11:51:56.455686 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:56.455667    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ad4893/sub-test-model-ad4893-kserve-698fb6869-cnk2j"]
Jul 17 11:51:57.658323 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:57.658294    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc5b3650-e267-438d-8a01-76dd7752723a" path="/var/lib/kubelet/pods/fc5b3650-e267-438d-8a01-76dd7752723a/volumes"
Jul 17 11:51:57.919455 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:57.919378    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:51:57.922361 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:57.922338    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:51:57.927988 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:57.927971    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:51:57.931079 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:57.931064    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sl2kk_ece5b685-5a25-414a-8b9a-d6de919e4f6c/ovn-acl-logging/0.log"
Jul 17 11:51:58.310150 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:58.310068    2566 scope.go:117] "RemoveContainer" containerID="1fe01f2a3c096c71d2ccaf65fc17bed3166e1f1830d5b5728b3a3194da89778f"
Jul 17 11:51:58.318375 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:51:58.318357    2566 scope.go:117] "RemoveContainer" containerID="a5967290215d8da0602717d37709ba1d76238de44fcadd43ce55a0f89db6b8ea"
Jul 17 11:52:04.651802 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:04.651770    2566 scope.go:117] "RemoveContainer" containerID="1ee88406ca8e52a8de7d0e265ad8eaa94ca732197077db2437929c026316f34e"
Jul 17 11:52:05.479098 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:05.479065    2566 generic.go:356] "Generic (PLEG): container finished" podID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerID="58828e9cd0b18d0131dec391af9e1138e4b6b03ef080e3e4315129f7ff38b3ac" exitCode=6
Jul 17 11:52:05.479292 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:05.479140    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" event={"ID":"0831eed5-422c-4eff-862b-3b2bab15ea12","Type":"ContainerDied","Data":"58828e9cd0b18d0131dec391af9e1138e4b6b03ef080e3e4315129f7ff38b3ac"}
Jul 17 11:52:05.479292 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:05.479245    2566 scope.go:117] "RemoveContainer" containerID="1ee88406ca8e52a8de7d0e265ad8eaa94ca732197077db2437929c026316f34e"
Jul 17 11:52:05.479606 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:05.479581    2566 scope.go:117] "RemoveContainer" containerID="58828e9cd0b18d0131dec391af9e1138e4b6b03ef080e3e4315129f7ff38b3ac"
Jul 17 11:52:05.479817 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:05.479796    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h_odh-ai-gateway-infra(0831eed5-422c-4eff-862b-3b2bab15ea12)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12"
Jul 17 11:52:06.507515 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:06.507481    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"]
Jul 17 11:52:06.646293 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:06.646271    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"
Jul 17 11:52:06.706626 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:06.706599    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0831eed5-422c-4eff-862b-3b2bab15ea12-maas-api-service-ca\") pod \"0831eed5-422c-4eff-862b-3b2bab15ea12\" (UID: \"0831eed5-422c-4eff-862b-3b2bab15ea12\") "
Jul 17 11:52:06.706993 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:06.706975    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0831eed5-422c-4eff-862b-3b2bab15ea12-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0831eed5-422c-4eff-862b-3b2bab15ea12" (UID: "0831eed5-422c-4eff-862b-3b2bab15ea12"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:52:06.707165 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:06.707151    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0831eed5-422c-4eff-862b-3b2bab15ea12-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:52:07.491341 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:07.491307    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"
Jul 17 11:52:07.491541 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:07.491342    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h" event={"ID":"0831eed5-422c-4eff-862b-3b2bab15ea12","Type":"ContainerDied","Data":"46c033ada66a44b09cfe1bce8a2f70f4926323eb783fd3019822f02838ce73b1"}
Jul 17 11:52:07.491541 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:07.491415    2566 scope.go:117] "RemoveContainer" containerID="58828e9cd0b18d0131dec391af9e1138e4b6b03ef080e3e4315129f7ff38b3ac"
Jul 17 11:52:07.516646 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:07.516622    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"]
Jul 17 11:52:07.521588 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:07.521568    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-l9q5h"]
Jul 17 11:52:07.657525 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:07.657485    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" path="/var/lib/kubelet/pods/0831eed5-422c-4eff-862b-3b2bab15ea12/volumes"
Jul 17 11:52:13.844446 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:13.844409    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:52:28.316356 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316325    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"]
Jul 17 11:52:28.316737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316661    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerName="revoke-keys"
Jul 17 11:52:28.316737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316671    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerName="revoke-keys"
Jul 17 11:52:28.316737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316684    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc5b3650-e267-438d-8a01-76dd7752723a" containerName="main"
Jul 17 11:52:28.316737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316690    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc5b3650-e267-438d-8a01-76dd7752723a" containerName="main"
Jul 17 11:52:28.316737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316720    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerName="revoke-keys"
Jul 17 11:52:28.316737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316726    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerName="revoke-keys"
Jul 17 11:52:28.316737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316733    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerName="revoke-keys"
Jul 17 11:52:28.316737 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316738    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerName="revoke-keys"
Jul 17 11:52:28.317008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316748    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerName="revoke-keys"
Jul 17 11:52:28.317008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316753    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerName="revoke-keys"
Jul 17 11:52:28.317008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316825    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerName="revoke-keys"
Jul 17 11:52:28.317008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316833    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc5b3650-e267-438d-8a01-76dd7752723a" containerName="main"
Jul 17 11:52:28.317008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316839    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e621f653-be26-4a66-b9b1-df8d88e380ad" containerName="revoke-keys"
Jul 17 11:52:28.317008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316846    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d036b439-faad-43f1-917e-665ebef960c3" containerName="revoke-keys"
Jul 17 11:52:28.317008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316851    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerName="revoke-keys"
Jul 17 11:52:28.317008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.316857    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0831eed5-422c-4eff-862b-3b2bab15ea12" containerName="revoke-keys"
Jul 17 11:52:28.319968 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.319954    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"
Jul 17 11:52:28.322871 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.322840    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:52:28.322871 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.322856    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:52:28.328740 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.328718    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"]
Jul 17 11:52:28.499942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.499911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f8875c-f7ff-43c6-92af-6b9ac8946541-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-8zcjh\" (UID: \"45f8875c-f7ff-43c6-92af-6b9ac8946541\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"
Jul 17 11:52:28.600501 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.600470    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f8875c-f7ff-43c6-92af-6b9ac8946541-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-8zcjh\" (UID: \"45f8875c-f7ff-43c6-92af-6b9ac8946541\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"
Jul 17 11:52:28.601342 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.601309    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f8875c-f7ff-43c6-92af-6b9ac8946541-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-8zcjh\" (UID: \"45f8875c-f7ff-43c6-92af-6b9ac8946541\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"
Jul 17 11:52:28.631340 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.631311    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"
Jul 17 11:52:28.764727 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:28.764677    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"]
Jul 17 11:52:28.766415 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:52:28.766375    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f8875c_f7ff_43c6_92af_6b9ac8946541.slice/crio-1c71b688a17c8b78eeba545e9b59ae7d689c0c7f17e0f8ca82a406b5578b83f9 WatchSource:0}: Error finding container 1c71b688a17c8b78eeba545e9b59ae7d689c0c7f17e0f8ca82a406b5578b83f9: Status 404 returned error can't find the container with id 1c71b688a17c8b78eeba545e9b59ae7d689c0c7f17e0f8ca82a406b5578b83f9
Jul 17 11:52:29.269803 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.269727    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"]
Jul 17 11:52:29.273276 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.273260    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"
Jul 17 11:52:29.280944 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.280815    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"]
Jul 17 11:52:29.409410 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.409378    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8014b9-d127-4d63-ac0b-0f29cbe98e70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk\" (UID: \"fe8014b9-d127-4d63-ac0b-0f29cbe98e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"
Jul 17 11:52:29.510787 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.510758    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8014b9-d127-4d63-ac0b-0f29cbe98e70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk\" (UID: \"fe8014b9-d127-4d63-ac0b-0f29cbe98e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"
Jul 17 11:52:29.511319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.511300    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8014b9-d127-4d63-ac0b-0f29cbe98e70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk\" (UID: \"fe8014b9-d127-4d63-ac0b-0f29cbe98e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"
Jul 17 11:52:29.585146 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.585126    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"
Jul 17 11:52:29.601520 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.601498    2566 generic.go:356] "Generic (PLEG): container finished" podID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerID="cf5029ea32a7e95c9e563ceb99f334b0a5e10b061fcb577df446ca150d797b2e" exitCode=6
Jul 17 11:52:29.601646 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.601527    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh" event={"ID":"45f8875c-f7ff-43c6-92af-6b9ac8946541","Type":"ContainerDied","Data":"cf5029ea32a7e95c9e563ceb99f334b0a5e10b061fcb577df446ca150d797b2e"}
Jul 17 11:52:29.601646 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.601595    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh" event={"ID":"45f8875c-f7ff-43c6-92af-6b9ac8946541","Type":"ContainerStarted","Data":"1c71b688a17c8b78eeba545e9b59ae7d689c0c7f17e0f8ca82a406b5578b83f9"}
Jul 17 11:52:29.601866 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.601851    2566 scope.go:117] "RemoveContainer" containerID="cf5029ea32a7e95c9e563ceb99f334b0a5e10b061fcb577df446ca150d797b2e"
Jul 17 11:52:29.923452 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:29.923426    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"]
Jul 17 11:52:29.925213 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:52:29.925190    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8014b9_d127_4d63_ac0b_0f29cbe98e70.slice/crio-1e37697a6bc3ce61fcc7cdf8683b00bb6b2e885979c14776872f4e75ea406cd6 WatchSource:0}: Error finding container 1e37697a6bc3ce61fcc7cdf8683b00bb6b2e885979c14776872f4e75ea406cd6: Status 404 returned error can't find the container with id 1e37697a6bc3ce61fcc7cdf8683b00bb6b2e885979c14776872f4e75ea406cd6
Jul 17 11:52:30.340055 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:30.340025    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:52:30.608272 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:30.608243    2566 generic.go:356] "Generic (PLEG): container finished" podID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerID="85c30dcd58e78f8063499caf15fab6e50bb15f251948dd93d3c2d38c36d8af84" exitCode=6
Jul 17 11:52:30.608635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:30.608328    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk" event={"ID":"fe8014b9-d127-4d63-ac0b-0f29cbe98e70","Type":"ContainerDied","Data":"85c30dcd58e78f8063499caf15fab6e50bb15f251948dd93d3c2d38c36d8af84"}
Jul 17 11:52:30.608635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:30.608374    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk" event={"ID":"fe8014b9-d127-4d63-ac0b-0f29cbe98e70","Type":"ContainerStarted","Data":"1e37697a6bc3ce61fcc7cdf8683b00bb6b2e885979c14776872f4e75ea406cd6"}
Jul 17 11:52:30.608635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:30.608503    2566 scope.go:117] "RemoveContainer" containerID="85c30dcd58e78f8063499caf15fab6e50bb15f251948dd93d3c2d38c36d8af84"
Jul 17 11:52:30.610296 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:30.610279    2566 generic.go:356] "Generic (PLEG): container finished" podID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerID="2c54cc5111c4122a3f04782f3a928fbab4a02f39f487d36eacad8d399ca53acb" exitCode=6
Jul 17 11:52:30.610416 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:30.610306    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh" event={"ID":"45f8875c-f7ff-43c6-92af-6b9ac8946541","Type":"ContainerDied","Data":"2c54cc5111c4122a3f04782f3a928fbab4a02f39f487d36eacad8d399ca53acb"}
Jul 17 11:52:30.610416 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:30.610342    2566 scope.go:117] "RemoveContainer" containerID="cf5029ea32a7e95c9e563ceb99f334b0a5e10b061fcb577df446ca150d797b2e"
Jul 17 11:52:30.610551 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:30.610531    2566 scope.go:117] "RemoveContainer" containerID="2c54cc5111c4122a3f04782f3a928fbab4a02f39f487d36eacad8d399ca53acb"
Jul 17 11:52:30.610793 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:30.610754    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-8zcjh_odh-ai-gateway-infra(45f8875c-f7ff-43c6-92af-6b9ac8946541)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541"
Jul 17 11:52:31.617817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:31.617790    2566 scope.go:117] "RemoveContainer" containerID="2c54cc5111c4122a3f04782f3a928fbab4a02f39f487d36eacad8d399ca53acb"
Jul 17 11:52:31.618234 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:31.618046    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-8zcjh_odh-ai-gateway-infra(45f8875c-f7ff-43c6-92af-6b9ac8946541)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541"
Jul 17 11:52:31.619387 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:31.619366    2566 generic.go:356] "Generic (PLEG): container finished" podID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerID="202bc0c40f5625b413707dcc8d73f9d2ded4b9443044aba201c8bd2e1b1c0270" exitCode=6
Jul 17 11:52:31.619494 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:31.619422    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk" event={"ID":"fe8014b9-d127-4d63-ac0b-0f29cbe98e70","Type":"ContainerDied","Data":"202bc0c40f5625b413707dcc8d73f9d2ded4b9443044aba201c8bd2e1b1c0270"}
Jul 17 11:52:31.619494 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:31.619475    2566 scope.go:117] "RemoveContainer" containerID="85c30dcd58e78f8063499caf15fab6e50bb15f251948dd93d3c2d38c36d8af84"
Jul 17 11:52:31.619608 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:31.619594    2566 scope.go:117] "RemoveContainer" containerID="202bc0c40f5625b413707dcc8d73f9d2ded4b9443044aba201c8bd2e1b1c0270"
Jul 17 11:52:31.619842 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:31.619821    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk_odh-ai-gateway-infra(fe8014b9-d127-4d63-ac0b-0f29cbe98e70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70"
Jul 17 11:52:32.625878 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:32.625854    2566 scope.go:117] "RemoveContainer" containerID="202bc0c40f5625b413707dcc8d73f9d2ded4b9443044aba201c8bd2e1b1c0270"
Jul 17 11:52:32.626334 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:32.626098    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk_odh-ai-gateway-infra(fe8014b9-d127-4d63-ac0b-0f29cbe98e70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70"
Jul 17 11:52:36.105691 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:36.105653    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"]
Jul 17 11:52:36.110775 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:36.110752    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"
Jul 17 11:52:36.115750 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:36.115719    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"]
Jul 17 11:52:36.163850 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:36.163818    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0212978d-95d1-4ebf-9f47-d287005c2cf0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-p7p8c\" (UID: \"0212978d-95d1-4ebf-9f47-d287005c2cf0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"
Jul 17 11:52:36.264845 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:36.264810    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0212978d-95d1-4ebf-9f47-d287005c2cf0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-p7p8c\" (UID: \"0212978d-95d1-4ebf-9f47-d287005c2cf0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"
Jul 17 11:52:36.265456 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:36.265438    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0212978d-95d1-4ebf-9f47-d287005c2cf0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-p7p8c\" (UID: \"0212978d-95d1-4ebf-9f47-d287005c2cf0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"
Jul 17 11:52:36.423383 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:36.423299    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"
Jul 17 11:52:36.555432 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:36.555405    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"]
Jul 17 11:52:36.557404 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:52:36.557371    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0212978d_95d1_4ebf_9f47_d287005c2cf0.slice/crio-64195611e9d79af55ae5fe51b7da13ca7443ad05ec50c0056c116f504cd01b72 WatchSource:0}: Error finding container 64195611e9d79af55ae5fe51b7da13ca7443ad05ec50c0056c116f504cd01b72: Status 404 returned error can't find the container with id 64195611e9d79af55ae5fe51b7da13ca7443ad05ec50c0056c116f504cd01b72
Jul 17 11:52:36.648418 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:36.648389    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c" event={"ID":"0212978d-95d1-4ebf-9f47-d287005c2cf0","Type":"ContainerStarted","Data":"64195611e9d79af55ae5fe51b7da13ca7443ad05ec50c0056c116f504cd01b72"}
Jul 17 11:52:37.655113 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:37.655071    2566 generic.go:356] "Generic (PLEG): container finished" podID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerID="b9af9ea76518101c4fc406e56f95d10a98751e5de74c1c735ceb862854cf791c" exitCode=6
Jul 17 11:52:37.655568 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:37.655315    2566 scope.go:117] "RemoveContainer" containerID="b9af9ea76518101c4fc406e56f95d10a98751e5de74c1c735ceb862854cf791c"
Jul 17 11:52:37.657971 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:37.657950    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c" event={"ID":"0212978d-95d1-4ebf-9f47-d287005c2cf0","Type":"ContainerDied","Data":"b9af9ea76518101c4fc406e56f95d10a98751e5de74c1c735ceb862854cf791c"}
Jul 17 11:52:38.661544 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:38.661442    2566 generic.go:356] "Generic (PLEG): container finished" podID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerID="39b352db290914caf66745889cabcce1cf51eeef43c6aa1b204c0514b283c737" exitCode=6
Jul 17 11:52:38.661544 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:38.661517    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c" event={"ID":"0212978d-95d1-4ebf-9f47-d287005c2cf0","Type":"ContainerDied","Data":"39b352db290914caf66745889cabcce1cf51eeef43c6aa1b204c0514b283c737"}
Jul 17 11:52:38.680080 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:38.661584    2566 scope.go:117] "RemoveContainer" containerID="b9af9ea76518101c4fc406e56f95d10a98751e5de74c1c735ceb862854cf791c"
Jul 17 11:52:38.680080 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:38.661779    2566 scope.go:117] "RemoveContainer" containerID="39b352db290914caf66745889cabcce1cf51eeef43c6aa1b204c0514b283c737"
Jul 17 11:52:38.680080 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:38.661987    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-p7p8c_odh-ai-gateway-infra(0212978d-95d1-4ebf-9f47-d287005c2cf0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0"
Jul 17 11:52:39.434889 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:39.434857    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"]
Jul 17 11:52:39.438117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:39.438098    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"
Jul 17 11:52:39.448403 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:39.448377    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"]
Jul 17 11:52:39.486345 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:39.486315    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531102a-487c-4890-b549-400cdf1ff752-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9\" (UID: \"7531102a-487c-4890-b549-400cdf1ff752\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"
Jul 17 11:52:39.587502 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:39.587473    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531102a-487c-4890-b549-400cdf1ff752-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9\" (UID: \"7531102a-487c-4890-b549-400cdf1ff752\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"
Jul 17 11:52:39.588072 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:39.588053    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531102a-487c-4890-b549-400cdf1ff752-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9\" (UID: \"7531102a-487c-4890-b549-400cdf1ff752\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"
Jul 17 11:52:39.667757 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:39.667731    2566 scope.go:117] "RemoveContainer" containerID="39b352db290914caf66745889cabcce1cf51eeef43c6aa1b204c0514b283c737"
Jul 17 11:52:39.668118 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:39.667939    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-p7p8c_odh-ai-gateway-infra(0212978d-95d1-4ebf-9f47-d287005c2cf0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0"
Jul 17 11:52:39.761025 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:39.760954    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"
Jul 17 11:52:39.890089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:39.890062    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"]
Jul 17 11:52:39.891239 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:52:39.891217    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7531102a_487c_4890_b549_400cdf1ff752.slice/crio-fbbe192b7bd41e7c8abc107aa7bc8716f17b746e4e952e94549385e57ef7ff01 WatchSource:0}: Error finding container fbbe192b7bd41e7c8abc107aa7bc8716f17b746e4e952e94549385e57ef7ff01: Status 404 returned error can't find the container with id fbbe192b7bd41e7c8abc107aa7bc8716f17b746e4e952e94549385e57ef7ff01
Jul 17 11:52:40.673847 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:40.673815    2566 generic.go:356] "Generic (PLEG): container finished" podID="7531102a-487c-4890-b549-400cdf1ff752" containerID="5ba0b4c449c8cbd5b8ff6feff7cf3341cb949baddd0e41edc99f2fb557356a0a" exitCode=6
Jul 17 11:52:40.674229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:40.673894    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9" event={"ID":"7531102a-487c-4890-b549-400cdf1ff752","Type":"ContainerDied","Data":"5ba0b4c449c8cbd5b8ff6feff7cf3341cb949baddd0e41edc99f2fb557356a0a"}
Jul 17 11:52:40.674229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:40.673940    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9" event={"ID":"7531102a-487c-4890-b549-400cdf1ff752","Type":"ContainerStarted","Data":"fbbe192b7bd41e7c8abc107aa7bc8716f17b746e4e952e94549385e57ef7ff01"}
Jul 17 11:52:40.674229 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:40.674023    2566 scope.go:117] "RemoveContainer" containerID="5ba0b4c449c8cbd5b8ff6feff7cf3341cb949baddd0e41edc99f2fb557356a0a"
Jul 17 11:52:41.680947 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:41.680917    2566 generic.go:356] "Generic (PLEG): container finished" podID="7531102a-487c-4890-b549-400cdf1ff752" containerID="ecadf7d1f4c3162524614e4b596da84878693a4158dd6cd7e003ff49054938af" exitCode=6
Jul 17 11:52:41.681376 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:41.681006    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9" event={"ID":"7531102a-487c-4890-b549-400cdf1ff752","Type":"ContainerDied","Data":"ecadf7d1f4c3162524614e4b596da84878693a4158dd6cd7e003ff49054938af"}
Jul 17 11:52:41.681376 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:41.681060    2566 scope.go:117] "RemoveContainer" containerID="5ba0b4c449c8cbd5b8ff6feff7cf3341cb949baddd0e41edc99f2fb557356a0a"
Jul 17 11:52:41.681376 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:41.681208    2566 scope.go:117] "RemoveContainer" containerID="ecadf7d1f4c3162524614e4b596da84878693a4158dd6cd7e003ff49054938af"
Jul 17 11:52:41.681556 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:41.681448    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9_odh-ai-gateway-infra(7531102a-487c-4890-b549-400cdf1ff752)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9" podUID="7531102a-487c-4890-b549-400cdf1ff752"
Jul 17 11:52:42.688307 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:42.688278    2566 scope.go:117] "RemoveContainer" containerID="ecadf7d1f4c3162524614e4b596da84878693a4158dd6cd7e003ff49054938af"
Jul 17 11:52:42.688753 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:42.688453    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9_odh-ai-gateway-infra(7531102a-487c-4890-b549-400cdf1ff752)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9" podUID="7531102a-487c-4890-b549-400cdf1ff752"
Jul 17 11:52:44.652240 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:44.652210    2566 scope.go:117] "RemoveContainer" containerID="2c54cc5111c4122a3f04782f3a928fbab4a02f39f487d36eacad8d399ca53acb"
Jul 17 11:52:44.652653 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:44.652331    2566 scope.go:117] "RemoveContainer" containerID="202bc0c40f5625b413707dcc8d73f9d2ded4b9443044aba201c8bd2e1b1c0270"
Jul 17 11:52:45.704699 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:45.704671    2566 generic.go:356] "Generic (PLEG): container finished" podID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerID="db20627a75eecd2fdd14fd85cb5d42fed25ea6739689442c5432394328dd5fc4" exitCode=6
Jul 17 11:52:45.705157 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:45.704740    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh" event={"ID":"45f8875c-f7ff-43c6-92af-6b9ac8946541","Type":"ContainerDied","Data":"db20627a75eecd2fdd14fd85cb5d42fed25ea6739689442c5432394328dd5fc4"}
Jul 17 11:52:45.705157 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:45.704797    2566 scope.go:117] "RemoveContainer" containerID="2c54cc5111c4122a3f04782f3a928fbab4a02f39f487d36eacad8d399ca53acb"
Jul 17 11:52:45.705157 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:45.705090    2566 scope.go:117] "RemoveContainer" containerID="db20627a75eecd2fdd14fd85cb5d42fed25ea6739689442c5432394328dd5fc4"
Jul 17 11:52:45.705346 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:45.705327    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-8zcjh_odh-ai-gateway-infra(45f8875c-f7ff-43c6-92af-6b9ac8946541)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541"
Jul 17 11:52:45.706650 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:45.706629    2566 generic.go:356] "Generic (PLEG): container finished" podID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerID="db7b0e0bed79222d6aaf9b9b1dc4cf3c650d7b7eb8d4beca859ca40aed4341a5" exitCode=6
Jul 17 11:52:45.706763 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:45.706697    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk" event={"ID":"fe8014b9-d127-4d63-ac0b-0f29cbe98e70","Type":"ContainerDied","Data":"db7b0e0bed79222d6aaf9b9b1dc4cf3c650d7b7eb8d4beca859ca40aed4341a5"}
Jul 17 11:52:45.706987 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:45.706973    2566 scope.go:117] "RemoveContainer" containerID="db7b0e0bed79222d6aaf9b9b1dc4cf3c650d7b7eb8d4beca859ca40aed4341a5"
Jul 17 11:52:45.707151 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:45.707136    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk_odh-ai-gateway-infra(fe8014b9-d127-4d63-ac0b-0f29cbe98e70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70"
Jul 17 11:52:45.716040 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:45.716022    2566 scope.go:117] "RemoveContainer" containerID="202bc0c40f5625b413707dcc8d73f9d2ded4b9443044aba201c8bd2e1b1c0270"
Jul 17 11:52:46.733755 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:46.733722    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"]
Jul 17 11:52:46.751800 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:46.751762    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"]
Jul 17 11:52:46.875337 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:46.875311    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"
Jul 17 11:52:46.910699 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:46.910676    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"
Jul 17 11:52:46.950767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:46.950741    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8014b9-d127-4d63-ac0b-0f29cbe98e70-maas-api-service-ca\") pod \"fe8014b9-d127-4d63-ac0b-0f29cbe98e70\" (UID: \"fe8014b9-d127-4d63-ac0b-0f29cbe98e70\") "
Jul 17 11:52:46.950893 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:46.950866    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f8875c-f7ff-43c6-92af-6b9ac8946541-maas-api-service-ca\") pod \"45f8875c-f7ff-43c6-92af-6b9ac8946541\" (UID: \"45f8875c-f7ff-43c6-92af-6b9ac8946541\") "
Jul 17 11:52:46.951138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:46.951111    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe8014b9-d127-4d63-ac0b-0f29cbe98e70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fe8014b9-d127-4d63-ac0b-0f29cbe98e70" (UID: "fe8014b9-d127-4d63-ac0b-0f29cbe98e70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:52:46.951220 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:46.951192    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45f8875c-f7ff-43c6-92af-6b9ac8946541-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "45f8875c-f7ff-43c6-92af-6b9ac8946541" (UID: "45f8875c-f7ff-43c6-92af-6b9ac8946541"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:52:46.951282 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:46.951218    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8014b9-d127-4d63-ac0b-0f29cbe98e70-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:52:47.051844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.051758    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f8875c-f7ff-43c6-92af-6b9ac8946541-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:52:47.721551 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.721529    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"
Jul 17 11:52:47.721551 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.721534    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk" event={"ID":"fe8014b9-d127-4d63-ac0b-0f29cbe98e70","Type":"ContainerDied","Data":"1e37697a6bc3ce61fcc7cdf8683b00bb6b2e885979c14776872f4e75ea406cd6"}
Jul 17 11:52:47.721767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.721591    2566 scope.go:117] "RemoveContainer" containerID="db7b0e0bed79222d6aaf9b9b1dc4cf3c650d7b7eb8d4beca859ca40aed4341a5"
Jul 17 11:52:47.723044 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.723018    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh" event={"ID":"45f8875c-f7ff-43c6-92af-6b9ac8946541","Type":"ContainerDied","Data":"1c71b688a17c8b78eeba545e9b59ae7d689c0c7f17e0f8ca82a406b5578b83f9"}
Jul 17 11:52:47.723168 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.723053    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"
Jul 17 11:52:47.730398 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.730380    2566 scope.go:117] "RemoveContainer" containerID="db20627a75eecd2fdd14fd85cb5d42fed25ea6739689442c5432394328dd5fc4"
Jul 17 11:52:47.741965 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.741943    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"]
Jul 17 11:52:47.747998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.747979    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-w4sqk"]
Jul 17 11:52:47.765612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.765588    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"]
Jul 17 11:52:47.774123 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:47.774100    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-8zcjh"]
Jul 17 11:52:49.657785 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:49.657754    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" path="/var/lib/kubelet/pods/45f8875c-f7ff-43c6-92af-6b9ac8946541/volumes"
Jul 17 11:52:49.658154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:49.658089    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" path="/var/lib/kubelet/pods/fe8014b9-d127-4d63-ac0b-0f29cbe98e70/volumes"
Jul 17 11:52:53.538464 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:53.538425    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:52:53.652276 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:53.652242    2566 scope.go:117] "RemoveContainer" containerID="39b352db290914caf66745889cabcce1cf51eeef43c6aa1b204c0514b283c737"
Jul 17 11:52:54.763313 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:54.763226    2566 generic.go:356] "Generic (PLEG): container finished" podID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerID="944eb0c0638bcf763d34ee2e78b089a29289581e9e1b066fd69bf8edcba2a18a" exitCode=6
Jul 17 11:52:54.763313 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:54.763294    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c" event={"ID":"0212978d-95d1-4ebf-9f47-d287005c2cf0","Type":"ContainerDied","Data":"944eb0c0638bcf763d34ee2e78b089a29289581e9e1b066fd69bf8edcba2a18a"}
Jul 17 11:52:54.763822 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:54.763354    2566 scope.go:117] "RemoveContainer" containerID="39b352db290914caf66745889cabcce1cf51eeef43c6aa1b204c0514b283c737"
Jul 17 11:52:54.763822 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:54.763624    2566 scope.go:117] "RemoveContainer" containerID="944eb0c0638bcf763d34ee2e78b089a29289581e9e1b066fd69bf8edcba2a18a"
Jul 17 11:52:54.763936 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:54.763874    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-p7p8c_odh-ai-gateway-infra(0212978d-95d1-4ebf-9f47-d287005c2cf0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0"
Jul 17 11:52:55.651927 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:55.651898    2566 scope.go:117] "RemoveContainer" containerID="ecadf7d1f4c3162524614e4b596da84878693a4158dd6cd7e003ff49054938af"
Jul 17 11:52:55.793233 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:55.793202    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"]
Jul 17 11:52:55.929935 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:55.929915    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"
Jul 17 11:52:56.031772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.031743    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0212978d-95d1-4ebf-9f47-d287005c2cf0-maas-api-service-ca\") pod \"0212978d-95d1-4ebf-9f47-d287005c2cf0\" (UID: \"0212978d-95d1-4ebf-9f47-d287005c2cf0\") "
Jul 17 11:52:56.032114 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.032090    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0212978d-95d1-4ebf-9f47-d287005c2cf0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0212978d-95d1-4ebf-9f47-d287005c2cf0" (UID: "0212978d-95d1-4ebf-9f47-d287005c2cf0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:52:56.133144 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.133119    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0212978d-95d1-4ebf-9f47-d287005c2cf0-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:52:56.783875 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.783790    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"
Jul 17 11:52:56.783875 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.783787    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c" event={"ID":"0212978d-95d1-4ebf-9f47-d287005c2cf0","Type":"ContainerDied","Data":"64195611e9d79af55ae5fe51b7da13ca7443ad05ec50c0056c116f504cd01b72"}
Jul 17 11:52:56.784120 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.783963    2566 scope.go:117] "RemoveContainer" containerID="944eb0c0638bcf763d34ee2e78b089a29289581e9e1b066fd69bf8edcba2a18a"
Jul 17 11:52:56.785784 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.785761    2566 generic.go:356] "Generic (PLEG): container finished" podID="7531102a-487c-4890-b549-400cdf1ff752" containerID="077833846a113ab8f659e7b21237b45a604211f914ef64b2b7d828a633e7ef63" exitCode=6
Jul 17 11:52:56.785897 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.785836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9" event={"ID":"7531102a-487c-4890-b549-400cdf1ff752","Type":"ContainerDied","Data":"077833846a113ab8f659e7b21237b45a604211f914ef64b2b7d828a633e7ef63"}
Jul 17 11:52:56.786177 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.786161    2566 scope.go:117] "RemoveContainer" containerID="077833846a113ab8f659e7b21237b45a604211f914ef64b2b7d828a633e7ef63"
Jul 17 11:52:56.786419 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:52:56.786398    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9_odh-ai-gateway-infra(7531102a-487c-4890-b549-400cdf1ff752)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9" podUID="7531102a-487c-4890-b549-400cdf1ff752"
Jul 17 11:52:56.793117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.793100    2566 scope.go:117] "RemoveContainer" containerID="ecadf7d1f4c3162524614e4b596da84878693a4158dd6cd7e003ff49054938af"
Jul 17 11:52:56.819759 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.819728    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"]
Jul 17 11:52:56.829751 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:56.829722    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-p7p8c"]
Jul 17 11:52:57.658296 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:57.658267    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" path="/var/lib/kubelet/pods/0212978d-95d1-4ebf-9f47-d287005c2cf0/volumes"
Jul 17 11:52:57.816347 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:57.816318    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"]
Jul 17 11:52:57.948595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:57.948573    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"
Jul 17 11:52:58.051946 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:58.051917    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531102a-487c-4890-b549-400cdf1ff752-maas-api-service-ca\") pod \"7531102a-487c-4890-b549-400cdf1ff752\" (UID: \"7531102a-487c-4890-b549-400cdf1ff752\") "
Jul 17 11:52:58.052332 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:58.052256    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7531102a-487c-4890-b549-400cdf1ff752-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7531102a-487c-4890-b549-400cdf1ff752" (UID: "7531102a-487c-4890-b549-400cdf1ff752"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:52:58.152969 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:58.152938    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531102a-487c-4890-b549-400cdf1ff752-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:52:58.799293 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:58.799218    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"
Jul 17 11:52:58.799293 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:58.799222    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9" event={"ID":"7531102a-487c-4890-b549-400cdf1ff752","Type":"ContainerDied","Data":"fbbe192b7bd41e7c8abc107aa7bc8716f17b746e4e952e94549385e57ef7ff01"}
Jul 17 11:52:58.799293 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:58.799285    2566 scope.go:117] "RemoveContainer" containerID="077833846a113ab8f659e7b21237b45a604211f914ef64b2b7d828a633e7ef63"
Jul 17 11:52:58.824434 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:58.824411    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"]
Jul 17 11:52:58.830138 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:58.830113    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vw2s9"]
Jul 17 11:52:59.657182 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:52:59.657152    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7531102a-487c-4890-b549-400cdf1ff752" path="/var/lib/kubelet/pods/7531102a-487c-4890-b549-400cdf1ff752/volumes"
Jul 17 11:53:02.965621 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.965591    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-139717857132080"]
Jul 17 11:53:02.966041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966027    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerName="revoke-keys"
Jul 17 11:53:02.966097 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966043    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerName="revoke-keys"
Jul 17 11:53:02.966097 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966052    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerName="revoke-keys"
Jul 17 11:53:02.966097 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966059    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerName="revoke-keys"
Jul 17 11:53:02.966097 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966069    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerName="revoke-keys"
Jul 17 11:53:02.966097 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966078    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerName="revoke-keys"
Jul 17 11:53:02.966097 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966086    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerName="revoke-keys"
Jul 17 11:53:02.966097 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966091    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966101    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7531102a-487c-4890-b549-400cdf1ff752" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966106    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7531102a-487c-4890-b549-400cdf1ff752" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966114    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966120    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966125    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966130    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966144    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966152    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966160    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7531102a-487c-4890-b549-400cdf1ff752" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966164    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7531102a-487c-4890-b549-400cdf1ff752" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966172    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966177    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966231    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966238    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7531102a-487c-4890-b549-400cdf1ff752" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966243    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966249    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966256    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7531102a-487c-4890-b549-400cdf1ff752" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966260    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966266    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966272    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerName="revoke-keys"
Jul 17 11:53:02.966305 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.966277    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="45f8875c-f7ff-43c6-92af-6b9ac8946541" containerName="revoke-keys"
Jul 17 11:53:02.970852 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.970834    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857132080"
Jul 17 11:53:02.973752 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.973726    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cvbgn\""
Jul 17 11:53:02.975653 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:02.975627    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-139717857132080"]
Jul 17 11:53:03.097772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:03.097739    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5wpl\" (UniqueName: \"kubernetes.io/projected/37748eec-d59b-4e94-b83a-74aaf16d1173-kube-api-access-m5wpl\") pod \"test-curl-7659-139717857132080\" (UID: \"37748eec-d59b-4e94-b83a-74aaf16d1173\") " pod="opendatahub/test-curl-7659-139717857132080"
Jul 17 11:53:03.198513 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:03.198480    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5wpl\" (UniqueName: \"kubernetes.io/projected/37748eec-d59b-4e94-b83a-74aaf16d1173-kube-api-access-m5wpl\") pod \"test-curl-7659-139717857132080\" (UID: \"37748eec-d59b-4e94-b83a-74aaf16d1173\") " pod="opendatahub/test-curl-7659-139717857132080"
Jul 17 11:53:03.208383 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:03.208346    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5wpl\" (UniqueName: \"kubernetes.io/projected/37748eec-d59b-4e94-b83a-74aaf16d1173-kube-api-access-m5wpl\") pod \"test-curl-7659-139717857132080\" (UID: \"37748eec-d59b-4e94-b83a-74aaf16d1173\") " pod="opendatahub/test-curl-7659-139717857132080"
Jul 17 11:53:03.281189 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:03.281123    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857132080"
Jul 17 11:53:03.407200 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:03.407173    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-139717857132080"]
Jul 17 11:53:03.409098 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:53:03.409063    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37748eec_d59b_4e94_b83a_74aaf16d1173.slice/crio-c428d054b7423bd4810911951da36545312b390ee370cbfdbe60d393168f7096 WatchSource:0}: Error finding container c428d054b7423bd4810911951da36545312b390ee370cbfdbe60d393168f7096: Status 404 returned error can't find the container with id c428d054b7423bd4810911951da36545312b390ee370cbfdbe60d393168f7096
Jul 17 11:53:03.410395 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:03.410375    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 11:53:03.827820 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:03.827775    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-139717857132080" event={"ID":"37748eec-d59b-4e94-b83a-74aaf16d1173","Type":"ContainerStarted","Data":"c428d054b7423bd4810911951da36545312b390ee370cbfdbe60d393168f7096"}
Jul 17 11:53:04.637401 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:04.637369    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:53:04.834939 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:04.834905    2566 generic.go:356] "Generic (PLEG): container finished" podID="37748eec-d59b-4e94-b83a-74aaf16d1173" containerID="31e27c88f1b3140152ada4418a10f6cd2f2995fdccab2d6f33f4e98640cb5dbb" exitCode=0
Jul 17 11:53:04.835117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:04.834995    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-139717857132080" event={"ID":"37748eec-d59b-4e94-b83a-74aaf16d1173","Type":"ContainerDied","Data":"31e27c88f1b3140152ada4418a10f6cd2f2995fdccab2d6f33f4e98640cb5dbb"}
Jul 17 11:53:05.974014 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:05.973983    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857132080"
Jul 17 11:53:06.011260 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.011231    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7659-139717857132080_37748eec-d59b-4e94-b83a-74aaf16d1173/test-curl-7659-139717857132080/0.log"
Jul 17 11:53:06.033361 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.033336    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-139717857132080"]
Jul 17 11:53:06.039829 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.039809    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-139717857132080"]
Jul 17 11:53:06.125861 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.125800    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5wpl\" (UniqueName: \"kubernetes.io/projected/37748eec-d59b-4e94-b83a-74aaf16d1173-kube-api-access-m5wpl\") pod \"37748eec-d59b-4e94-b83a-74aaf16d1173\" (UID: \"37748eec-d59b-4e94-b83a-74aaf16d1173\") "
Jul 17 11:53:06.127936 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.127908    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37748eec-d59b-4e94-b83a-74aaf16d1173-kube-api-access-m5wpl" (OuterVolumeSpecName: "kube-api-access-m5wpl") pod "37748eec-d59b-4e94-b83a-74aaf16d1173" (UID: "37748eec-d59b-4e94-b83a-74aaf16d1173"). InnerVolumeSpecName "kube-api-access-m5wpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:53:06.210959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.210928    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:06.211283 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211271    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerName="revoke-keys"
Jul 17 11:53:06.211328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211285    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerName="revoke-keys"
Jul 17 11:53:06.211328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211295    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7531102a-487c-4890-b549-400cdf1ff752" containerName="revoke-keys"
Jul 17 11:53:06.211328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211300    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7531102a-487c-4890-b549-400cdf1ff752" containerName="revoke-keys"
Jul 17 11:53:06.211328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211317    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37748eec-d59b-4e94-b83a-74aaf16d1173" containerName="test-curl-7659-139717857132080"
Jul 17 11:53:06.211328 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211324    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="37748eec-d59b-4e94-b83a-74aaf16d1173" containerName="test-curl-7659-139717857132080"
Jul 17 11:53:06.211479 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211390    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe8014b9-d127-4d63-ac0b-0f29cbe98e70" containerName="revoke-keys"
Jul 17 11:53:06.211479 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211400    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7531102a-487c-4890-b549-400cdf1ff752" containerName="revoke-keys"
Jul 17 11:53:06.211479 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211407    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0212978d-95d1-4ebf-9f47-d287005c2cf0" containerName="revoke-keys"
Jul 17 11:53:06.211479 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.211414    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="37748eec-d59b-4e94-b83a-74aaf16d1173" containerName="test-curl-7659-139717857132080"
Jul 17 11:53:06.214435 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.214421    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:06.222524 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.222503    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:06.227068 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.227050    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5wpl\" (UniqueName: \"kubernetes.io/projected/37748eec-d59b-4e94-b83a-74aaf16d1173-kube-api-access-m5wpl\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:53:06.328041 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.328015    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnwdc\" (UniqueName: \"kubernetes.io/projected/79098ac1-8680-425d-840f-c6df2584d1b1-kube-api-access-jnwdc\") pod \"test-curl-7659-139717857093648\" (UID: \"79098ac1-8680-425d-840f-c6df2584d1b1\") " pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:06.429421 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.429344    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnwdc\" (UniqueName: \"kubernetes.io/projected/79098ac1-8680-425d-840f-c6df2584d1b1-kube-api-access-jnwdc\") pod \"test-curl-7659-139717857093648\" (UID: \"79098ac1-8680-425d-840f-c6df2584d1b1\") " pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:06.438855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.438829    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnwdc\" (UniqueName: \"kubernetes.io/projected/79098ac1-8680-425d-840f-c6df2584d1b1-kube-api-access-jnwdc\") pod \"test-curl-7659-139717857093648\" (UID: \"79098ac1-8680-425d-840f-c6df2584d1b1\") " pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:06.523950 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.523924    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:06.648779 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.648757    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:06.650798 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:53:06.650772    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79098ac1_8680_425d_840f_c6df2584d1b1.slice/crio-f3ed0933da989622888f2069997b14843cdbee78ffe5ae6109194549ae081926 WatchSource:0}: Error finding container f3ed0933da989622888f2069997b14843cdbee78ffe5ae6109194549ae081926: Status 404 returned error can't find the container with id f3ed0933da989622888f2069997b14843cdbee78ffe5ae6109194549ae081926
Jul 17 11:53:06.846069 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.846046    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857132080"
Jul 17 11:53:06.846069 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.846054    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c428d054b7423bd4810911951da36545312b390ee370cbfdbe60d393168f7096"
Jul 17 11:53:06.847418 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:06.847389    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-139717857093648" event={"ID":"79098ac1-8680-425d-840f-c6df2584d1b1","Type":"ContainerStarted","Data":"f3ed0933da989622888f2069997b14843cdbee78ffe5ae6109194549ae081926"}
Jul 17 11:53:06.856488 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:06.856462    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7659-139717857132080\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="37748eec-d59b-4e94-b83a-74aaf16d1173" pod="opendatahub/test-curl-7659-139717857132080"
Jul 17 11:53:07.657523 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:07.657487    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7659-139717857132080\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="37748eec-d59b-4e94-b83a-74aaf16d1173" pod="opendatahub/test-curl-7659-139717857132080"
Jul 17 11:53:07.659055 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:07.659035    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37748eec-d59b-4e94-b83a-74aaf16d1173" path="/var/lib/kubelet/pods/37748eec-d59b-4e94-b83a-74aaf16d1173/volumes"
Jul 17 11:53:07.854183 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:07.854154    2566 generic.go:356] "Generic (PLEG): container finished" podID="79098ac1-8680-425d-840f-c6df2584d1b1" containerID="dde571f4223f53df0b0f3091cfa1bffd73198707a408815307d97af6e4273d62" exitCode=0
Jul 17 11:53:07.854337 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:07.854237    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-139717857093648" event={"ID":"79098ac1-8680-425d-840f-c6df2584d1b1","Type":"ContainerDied","Data":"dde571f4223f53df0b0f3091cfa1bffd73198707a408815307d97af6e4273d62"}
Jul 17 11:53:08.992154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:08.992132    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:09.017917 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.017896    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7659-139717857093648_79098ac1-8680-425d-840f-c6df2584d1b1/test-curl-7659-139717857093648/0.log"
Jul 17 11:53:09.041917 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.041895    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:09.046672 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.046647    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:09.154386 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.154329    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnwdc\" (UniqueName: \"kubernetes.io/projected/79098ac1-8680-425d-840f-c6df2584d1b1-kube-api-access-jnwdc\") pod \"79098ac1-8680-425d-840f-c6df2584d1b1\" (UID: \"79098ac1-8680-425d-840f-c6df2584d1b1\") "
Jul 17 11:53:09.156366 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.156346    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79098ac1-8680-425d-840f-c6df2584d1b1-kube-api-access-jnwdc" (OuterVolumeSpecName: "kube-api-access-jnwdc") pod "79098ac1-8680-425d-840f-c6df2584d1b1" (UID: "79098ac1-8680-425d-840f-c6df2584d1b1"). InnerVolumeSpecName "kube-api-access-jnwdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:53:09.174816 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.174793    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:09.175172 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.175160    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79098ac1-8680-425d-840f-c6df2584d1b1" containerName="test-curl-7659-139717857093648"
Jul 17 11:53:09.175218 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.175174    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="79098ac1-8680-425d-840f-c6df2584d1b1" containerName="test-curl-7659-139717857093648"
Jul 17 11:53:09.175254 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.175242    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="79098ac1-8680-425d-840f-c6df2584d1b1" containerName="test-curl-7659-139717857093648"
Jul 17 11:53:09.178136 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.178122    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:09.185757 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.185738    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:09.255862 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.255833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwg87\" (UniqueName: \"kubernetes.io/projected/f2f324e3-d7b0-4e15-97ca-87a1b2ef7234-kube-api-access-rwg87\") pod \"test-curl-7659-139717857093648\" (UID: \"f2f324e3-d7b0-4e15-97ca-87a1b2ef7234\") " pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:09.255989 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.255881    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnwdc\" (UniqueName: \"kubernetes.io/projected/79098ac1-8680-425d-840f-c6df2584d1b1-kube-api-access-jnwdc\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:53:09.356357 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.356325    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwg87\" (UniqueName: \"kubernetes.io/projected/f2f324e3-d7b0-4e15-97ca-87a1b2ef7234-kube-api-access-rwg87\") pod \"test-curl-7659-139717857093648\" (UID: \"f2f324e3-d7b0-4e15-97ca-87a1b2ef7234\") " pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:09.366090 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.366064    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwg87\" (UniqueName: \"kubernetes.io/projected/f2f324e3-d7b0-4e15-97ca-87a1b2ef7234-kube-api-access-rwg87\") pod \"test-curl-7659-139717857093648\" (UID: \"f2f324e3-d7b0-4e15-97ca-87a1b2ef7234\") " pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:09.487168 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.487112    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:09.612451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.612428    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:09.615036 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:53:09.615009    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2f324e3_d7b0_4e15_97ca_87a1b2ef7234.slice/crio-c0c3adf5c5f537a2a6803eec50840eccc7c3284cee3ea15052806ed8f720aeb9 WatchSource:0}: Error finding container c0c3adf5c5f537a2a6803eec50840eccc7c3284cee3ea15052806ed8f720aeb9: Status 404 returned error can't find the container with id c0c3adf5c5f537a2a6803eec50840eccc7c3284cee3ea15052806ed8f720aeb9
Jul 17 11:53:09.657637 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.657613    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79098ac1-8680-425d-840f-c6df2584d1b1" path="/var/lib/kubelet/pods/79098ac1-8680-425d-840f-c6df2584d1b1/volumes"
Jul 17 11:53:09.865627 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.865590    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-139717857093648" event={"ID":"f2f324e3-d7b0-4e15-97ca-87a1b2ef7234","Type":"ContainerStarted","Data":"c0c3adf5c5f537a2a6803eec50840eccc7c3284cee3ea15052806ed8f720aeb9"}
Jul 17 11:53:09.866817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.866798    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:09.866950 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:09.866850    2566 scope.go:117] "RemoveContainer" containerID="dde571f4223f53df0b0f3091cfa1bffd73198707a408815307d97af6e4273d62"
Jul 17 11:53:10.875060 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:10.875027    2566 generic.go:356] "Generic (PLEG): container finished" podID="f2f324e3-d7b0-4e15-97ca-87a1b2ef7234" containerID="ebdddef02bbcbe7145477483ec8a2ed86b5796402715a9c55746d8627d553ffd" exitCode=0
Jul 17 11:53:10.875480 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:10.875116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-139717857093648" event={"ID":"f2f324e3-d7b0-4e15-97ca-87a1b2ef7234","Type":"ContainerDied","Data":"ebdddef02bbcbe7145477483ec8a2ed86b5796402715a9c55746d8627d553ffd"}
Jul 17 11:53:12.022089 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:12.022066    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:12.042098 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:12.042076    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7659-139717857093648_f2f324e3-d7b0-4e15-97ca-87a1b2ef7234/test-curl-7659-139717857093648/0.log"
Jul 17 11:53:12.067769 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:12.067742    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:12.072595 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:12.072574    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-139717857093648"]
Jul 17 11:53:12.181883 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:12.181816    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwg87\" (UniqueName: \"kubernetes.io/projected/f2f324e3-d7b0-4e15-97ca-87a1b2ef7234-kube-api-access-rwg87\") pod \"f2f324e3-d7b0-4e15-97ca-87a1b2ef7234\" (UID: \"f2f324e3-d7b0-4e15-97ca-87a1b2ef7234\") "
Jul 17 11:53:12.183973 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:12.183951    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f324e3-d7b0-4e15-97ca-87a1b2ef7234-kube-api-access-rwg87" (OuterVolumeSpecName: "kube-api-access-rwg87") pod "f2f324e3-d7b0-4e15-97ca-87a1b2ef7234" (UID: "f2f324e3-d7b0-4e15-97ca-87a1b2ef7234"). InnerVolumeSpecName "kube-api-access-rwg87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:53:12.283190 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:12.283156    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwg87\" (UniqueName: \"kubernetes.io/projected/f2f324e3-d7b0-4e15-97ca-87a1b2ef7234-kube-api-access-rwg87\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:53:12.888970 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:12.888941    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:12.889159 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:12.888990    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0c3adf5c5f537a2a6803eec50840eccc7c3284cee3ea15052806ed8f720aeb9"
Jul 17 11:53:12.898745 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:12.898697    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7659-139717857093648\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="f2f324e3-d7b0-4e15-97ca-87a1b2ef7234" pod="opendatahub/test-curl-7659-139717857093648"
Jul 17 11:53:13.657359 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:13.657327    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2f324e3-d7b0-4e15-97ca-87a1b2ef7234" path="/var/lib/kubelet/pods/f2f324e3-d7b0-4e15-97ca-87a1b2ef7234/volumes"
Jul 17 11:53:18.109560 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.109523    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"]
Jul 17 11:53:18.109964 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.109889    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f324e3-d7b0-4e15-97ca-87a1b2ef7234" containerName="test-curl-7659-139717857093648"
Jul 17 11:53:18.109964 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.109902    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f324e3-d7b0-4e15-97ca-87a1b2ef7234" containerName="test-curl-7659-139717857093648"
Jul 17 11:53:18.110044 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.109972    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f324e3-d7b0-4e15-97ca-87a1b2ef7234" containerName="test-curl-7659-139717857093648"
Jul 17 11:53:18.113068 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.113052    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"
Jul 17 11:53:18.115659 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.115635    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:53:18.115810 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.115634    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:53:18.120649 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.120626    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"]
Jul 17 11:53:18.236005 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.235968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1337966a-dabf-439d-9ff6-0b3c54842a80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ad4893-zf667\" (UID: \"1337966a-dabf-439d-9ff6-0b3c54842a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"
Jul 17 11:53:18.336798 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.336759    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1337966a-dabf-439d-9ff6-0b3c54842a80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ad4893-zf667\" (UID: \"1337966a-dabf-439d-9ff6-0b3c54842a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"
Jul 17 11:53:18.337381 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.337362    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1337966a-dabf-439d-9ff6-0b3c54842a80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ad4893-zf667\" (UID: \"1337966a-dabf-439d-9ff6-0b3c54842a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"
Jul 17 11:53:18.362760 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.362684    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"]
Jul 17 11:53:18.365994 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.365979    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"
Jul 17 11:53:18.374581 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.374558    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"]
Jul 17 11:53:18.423873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.423843    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"
Jul 17 11:53:18.539310 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.539276    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8478c3c8-d869-4b3e-9238-639242fdcd9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-qvv9h\" (UID: \"8478c3c8-d869-4b3e-9238-639242fdcd9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"
Jul 17 11:53:18.553091 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.553064    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"]
Jul 17 11:53:18.554957 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:53:18.554937    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1337966a_dabf_439d_9ff6_0b3c54842a80.slice/crio-7d4af54e17400f7893719933d14d6c68ebb514e203c799cca0ba9cd5c8dc9ce1 WatchSource:0}: Error finding container 7d4af54e17400f7893719933d14d6c68ebb514e203c799cca0ba9cd5c8dc9ce1: Status 404 returned error can't find the container with id 7d4af54e17400f7893719933d14d6c68ebb514e203c799cca0ba9cd5c8dc9ce1
Jul 17 11:53:18.640090 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.640030    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8478c3c8-d869-4b3e-9238-639242fdcd9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-qvv9h\" (UID: \"8478c3c8-d869-4b3e-9238-639242fdcd9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"
Jul 17 11:53:18.640556 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.640539    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8478c3c8-d869-4b3e-9238-639242fdcd9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-45989be5-qvv9h\" (UID: \"8478c3c8-d869-4b3e-9238-639242fdcd9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"
Jul 17 11:53:18.677188 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.677170    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"
Jul 17 11:53:18.809694 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.809671    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"]
Jul 17 11:53:18.811844 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:53:18.811816    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8478c3c8_d869_4b3e_9238_639242fdcd9f.slice/crio-51fc49bccbbf0c2b22363560c7d6bdcbec3fa3d0af379d4749b7572d024a7d6d WatchSource:0}: Error finding container 51fc49bccbbf0c2b22363560c7d6bdcbec3fa3d0af379d4749b7572d024a7d6d: Status 404 returned error can't find the container with id 51fc49bccbbf0c2b22363560c7d6bdcbec3fa3d0af379d4749b7572d024a7d6d
Jul 17 11:53:18.924596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.924512    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667" event={"ID":"1337966a-dabf-439d-9ff6-0b3c54842a80","Type":"ContainerStarted","Data":"d0579f8f09e205251f79297296a878e13e5c68708f22ea67c9c1d29cf493dd5f"}
Jul 17 11:53:18.924596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.924561    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667" event={"ID":"1337966a-dabf-439d-9ff6-0b3c54842a80","Type":"ContainerStarted","Data":"7d4af54e17400f7893719933d14d6c68ebb514e203c799cca0ba9cd5c8dc9ce1"}
Jul 17 11:53:18.925858 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.925836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" event={"ID":"8478c3c8-d869-4b3e-9238-639242fdcd9f","Type":"ContainerStarted","Data":"51fc49bccbbf0c2b22363560c7d6bdcbec3fa3d0af379d4749b7572d024a7d6d"}
Jul 17 11:53:18.943920 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:18.943878    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667" podStartSLOduration=0.943863982 podStartE2EDuration="943.863982ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:53:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:53:18.941578565 +0000 UTC m=+3381.798662511" watchObservedRunningTime="2026-07-17 11:53:18.943863982 +0000 UTC m=+3381.800947927"
Jul 17 11:53:19.319046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.318972    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"]
Jul 17 11:53:19.327860 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.327498    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"
Jul 17 11:53:19.331695 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.331661    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"]
Jul 17 11:53:19.447823 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.447796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b63803-4a40-48c9-867d-c5f74c389116-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd\" (UID: \"02b63803-4a40-48c9-867d-c5f74c389116\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"
Jul 17 11:53:19.549333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.549309    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b63803-4a40-48c9-867d-c5f74c389116-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd\" (UID: \"02b63803-4a40-48c9-867d-c5f74c389116\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"
Jul 17 11:53:19.550043 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.550026    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b63803-4a40-48c9-867d-c5f74c389116-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd\" (UID: \"02b63803-4a40-48c9-867d-c5f74c389116\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"
Jul 17 11:53:19.690890 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.690864    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"
Jul 17 11:53:19.824017 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.823995    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"]
Jul 17 11:53:19.825774 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:53:19.825749    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b63803_4a40_48c9_867d_c5f74c389116.slice/crio-b43cfbbca37d92600f044f1dc5e87becca53764fa18fb42f697c01ed474f4766 WatchSource:0}: Error finding container b43cfbbca37d92600f044f1dc5e87becca53764fa18fb42f697c01ed474f4766: Status 404 returned error can't find the container with id b43cfbbca37d92600f044f1dc5e87becca53764fa18fb42f697c01ed474f4766
Jul 17 11:53:19.932131 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.932096    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" event={"ID":"02b63803-4a40-48c9-867d-c5f74c389116","Type":"ContainerStarted","Data":"b43cfbbca37d92600f044f1dc5e87becca53764fa18fb42f697c01ed474f4766"}
Jul 17 11:53:19.933445 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.933418    2566 generic.go:356] "Generic (PLEG): container finished" podID="1337966a-dabf-439d-9ff6-0b3c54842a80" containerID="d0579f8f09e205251f79297296a878e13e5c68708f22ea67c9c1d29cf493dd5f" exitCode=0
Jul 17 11:53:19.933559 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.933506    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667" event={"ID":"1337966a-dabf-439d-9ff6-0b3c54842a80","Type":"ContainerDied","Data":"d0579f8f09e205251f79297296a878e13e5c68708f22ea67c9c1d29cf493dd5f"}
Jul 17 11:53:19.934841 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.934819    2566 generic.go:356] "Generic (PLEG): container finished" podID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerID="bac9044025dd529bd8f0efba46f0b349695151137e356d3c32c9d8f50fa43c56" exitCode=6
Jul 17 11:53:19.934933 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.934852    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" event={"ID":"8478c3c8-d869-4b3e-9238-639242fdcd9f","Type":"ContainerDied","Data":"bac9044025dd529bd8f0efba46f0b349695151137e356d3c32c9d8f50fa43c56"}
Jul 17 11:53:19.935044 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:19.935030    2566 scope.go:117] "RemoveContainer" containerID="bac9044025dd529bd8f0efba46f0b349695151137e356d3c32c9d8f50fa43c56"
Jul 17 11:53:20.941977 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:20.941949    2566 generic.go:356] "Generic (PLEG): container finished" podID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerID="72e32c077337cf657220745de3a78aa6de4c95189b6d81003b44c88562d2d29e" exitCode=6
Jul 17 11:53:20.942451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:20.942020    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" event={"ID":"8478c3c8-d869-4b3e-9238-639242fdcd9f","Type":"ContainerDied","Data":"72e32c077337cf657220745de3a78aa6de4c95189b6d81003b44c88562d2d29e"}
Jul 17 11:53:20.942451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:20.942077    2566 scope.go:117] "RemoveContainer" containerID="bac9044025dd529bd8f0efba46f0b349695151137e356d3c32c9d8f50fa43c56"
Jul 17 11:53:20.942451 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:20.942287    2566 scope.go:117] "RemoveContainer" containerID="72e32c077337cf657220745de3a78aa6de4c95189b6d81003b44c88562d2d29e"
Jul 17 11:53:20.942619 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:20.942511    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-qvv9h_odh-ai-gateway-infra(8478c3c8-d869-4b3e-9238-639242fdcd9f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f"
Jul 17 11:53:20.943626 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:20.943603    2566 generic.go:356] "Generic (PLEG): container finished" podID="02b63803-4a40-48c9-867d-c5f74c389116" containerID="44617e8b3c02eecb10db43e498618f9d6a95c711be1d602af491fe1be848cc1a" exitCode=6
Jul 17 11:53:20.943761 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:20.943681    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" event={"ID":"02b63803-4a40-48c9-867d-c5f74c389116","Type":"ContainerDied","Data":"44617e8b3c02eecb10db43e498618f9d6a95c711be1d602af491fe1be848cc1a"}
Jul 17 11:53:20.943882 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:20.943864    2566 scope.go:117] "RemoveContainer" containerID="44617e8b3c02eecb10db43e498618f9d6a95c711be1d602af491fe1be848cc1a"
Jul 17 11:53:21.072383 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.072360    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"
Jul 17 11:53:21.074370 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.074348    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1337966a-dabf-439d-9ff6-0b3c54842a80-maas-api-service-ca\") pod \"1337966a-dabf-439d-9ff6-0b3c54842a80\" (UID: \"1337966a-dabf-439d-9ff6-0b3c54842a80\") "
Jul 17 11:53:21.074668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.074645    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1337966a-dabf-439d-9ff6-0b3c54842a80-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1337966a-dabf-439d-9ff6-0b3c54842a80" (UID: "1337966a-dabf-439d-9ff6-0b3c54842a80"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:53:21.175696 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.175668    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1337966a-dabf-439d-9ff6-0b3c54842a80-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:53:21.952767 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.952736    2566 generic.go:356] "Generic (PLEG): container finished" podID="02b63803-4a40-48c9-867d-c5f74c389116" containerID="05946bcaad8203f78a62fca02032c4620a8cdfecfbc0dcc460ab708ead69b79c" exitCode=6
Jul 17 11:53:21.953191 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.952823    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" event={"ID":"02b63803-4a40-48c9-867d-c5f74c389116","Type":"ContainerDied","Data":"05946bcaad8203f78a62fca02032c4620a8cdfecfbc0dcc460ab708ead69b79c"}
Jul 17 11:53:21.953191 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.952885    2566 scope.go:117] "RemoveContainer" containerID="44617e8b3c02eecb10db43e498618f9d6a95c711be1d602af491fe1be848cc1a"
Jul 17 11:53:21.953191 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.952988    2566 scope.go:117] "RemoveContainer" containerID="05946bcaad8203f78a62fca02032c4620a8cdfecfbc0dcc460ab708ead69b79c"
Jul 17 11:53:21.953374 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:21.953216    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd_odh-ai-gateway-infra(02b63803-4a40-48c9-867d-c5f74c389116)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" podUID="02b63803-4a40-48c9-867d-c5f74c389116"
Jul 17 11:53:21.954273 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.954256    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667"
Jul 17 11:53:21.954333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.954280    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ad4893-zf667" event={"ID":"1337966a-dabf-439d-9ff6-0b3c54842a80","Type":"ContainerDied","Data":"7d4af54e17400f7893719933d14d6c68ebb514e203c799cca0ba9cd5c8dc9ce1"}
Jul 17 11:53:21.954333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.954319    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d4af54e17400f7893719933d14d6c68ebb514e203c799cca0ba9cd5c8dc9ce1"
Jul 17 11:53:21.956061 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:21.956033    2566 scope.go:117] "RemoveContainer" containerID="72e32c077337cf657220745de3a78aa6de4c95189b6d81003b44c88562d2d29e"
Jul 17 11:53:21.956255 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:21.956234    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-qvv9h_odh-ai-gateway-infra(8478c3c8-d869-4b3e-9238-639242fdcd9f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f"
Jul 17 11:53:22.963007 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:22.962975    2566 scope.go:117] "RemoveContainer" containerID="05946bcaad8203f78a62fca02032c4620a8cdfecfbc0dcc460ab708ead69b79c"
Jul 17 11:53:22.963454 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:22.963215    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd_odh-ai-gateway-infra(02b63803-4a40-48c9-867d-c5f74c389116)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" podUID="02b63803-4a40-48c9-867d-c5f74c389116"
Jul 17 11:53:29.480393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.480355    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"]
Jul 17 11:53:29.480950 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.480931    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1337966a-dabf-439d-9ff6-0b3c54842a80" containerName="revoke-keys"
Jul 17 11:53:29.481033 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.480953    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1337966a-dabf-439d-9ff6-0b3c54842a80" containerName="revoke-keys"
Jul 17 11:53:29.481092 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.481044    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1337966a-dabf-439d-9ff6-0b3c54842a80" containerName="revoke-keys"
Jul 17 11:53:29.485392 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.485370    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"
Jul 17 11:53:29.493149 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.493030    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"]
Jul 17 11:53:29.527686 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.527658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd459aa-5b85-42a1-aea1-dfb15326de5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj\" (UID: \"5dd459aa-5b85-42a1-aea1-dfb15326de5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"
Jul 17 11:53:29.629081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.629037    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd459aa-5b85-42a1-aea1-dfb15326de5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj\" (UID: \"5dd459aa-5b85-42a1-aea1-dfb15326de5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"
Jul 17 11:53:29.629643 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.629624    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd459aa-5b85-42a1-aea1-dfb15326de5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj\" (UID: \"5dd459aa-5b85-42a1-aea1-dfb15326de5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"
Jul 17 11:53:29.797329 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.797232    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"
Jul 17 11:53:29.932107 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:29.932066    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"]
Jul 17 11:53:29.932924 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:53:29.932890    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd459aa_5b85_42a1_aea1_dfb15326de5c.slice/crio-77cb6f1cf0bc675cfb84fb58c573b3e5240d060383426195de880d59297332ca WatchSource:0}: Error finding container 77cb6f1cf0bc675cfb84fb58c573b3e5240d060383426195de880d59297332ca: Status 404 returned error can't find the container with id 77cb6f1cf0bc675cfb84fb58c573b3e5240d060383426195de880d59297332ca
Jul 17 11:53:30.000095 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:30.000066    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" event={"ID":"5dd459aa-5b85-42a1-aea1-dfb15326de5c","Type":"ContainerStarted","Data":"77cb6f1cf0bc675cfb84fb58c573b3e5240d060383426195de880d59297332ca"}
Jul 17 11:53:31.006888 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:31.006852    2566 generic.go:356] "Generic (PLEG): container finished" podID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerID="ec836e989bbf44149e66ab163f70571644780a1a809e4491ffb8b6c3ce6f44be" exitCode=6
Jul 17 11:53:31.007247 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:31.006918    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" event={"ID":"5dd459aa-5b85-42a1-aea1-dfb15326de5c","Type":"ContainerDied","Data":"ec836e989bbf44149e66ab163f70571644780a1a809e4491ffb8b6c3ce6f44be"}
Jul 17 11:53:31.007247 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:31.007113    2566 scope.go:117] "RemoveContainer" containerID="ec836e989bbf44149e66ab163f70571644780a1a809e4491ffb8b6c3ce6f44be"
Jul 17 11:53:32.013772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:32.013658    2566 generic.go:356] "Generic (PLEG): container finished" podID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerID="10139e4c3080b30c7442abaee203fee90609f47b1b8b110311b6c5862b67f31b" exitCode=6
Jul 17 11:53:32.013772 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:32.013740    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" event={"ID":"5dd459aa-5b85-42a1-aea1-dfb15326de5c","Type":"ContainerDied","Data":"10139e4c3080b30c7442abaee203fee90609f47b1b8b110311b6c5862b67f31b"}
Jul 17 11:53:32.014253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:32.013805    2566 scope.go:117] "RemoveContainer" containerID="ec836e989bbf44149e66ab163f70571644780a1a809e4491ffb8b6c3ce6f44be"
Jul 17 11:53:32.014253 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:32.014086    2566 scope.go:117] "RemoveContainer" containerID="10139e4c3080b30c7442abaee203fee90609f47b1b8b110311b6c5862b67f31b"
Jul 17 11:53:32.014382 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:32.014297    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj_odh-ai-gateway-infra(5dd459aa-5b85-42a1-aea1-dfb15326de5c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c"
Jul 17 11:53:33.021333 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:33.021310    2566 scope.go:117] "RemoveContainer" containerID="10139e4c3080b30c7442abaee203fee90609f47b1b8b110311b6c5862b67f31b"
Jul 17 11:53:33.021718 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:33.021524    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj_odh-ai-gateway-infra(5dd459aa-5b85-42a1-aea1-dfb15326de5c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c"
Jul 17 11:53:33.144625 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:33.144589    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:53:34.652065 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:34.652036    2566 scope.go:117] "RemoveContainer" containerID="72e32c077337cf657220745de3a78aa6de4c95189b6d81003b44c88562d2d29e"
Jul 17 11:53:35.033143 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:35.033060    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" event={"ID":"8478c3c8-d869-4b3e-9238-639242fdcd9f","Type":"ContainerStarted","Data":"1719fe82e4ffd452d863b2aca9b65dd57e61fd9afac52d72f03c4d552a9332d4"}
Jul 17 11:53:35.051300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:35.051254    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" podStartSLOduration=17.051235405 podStartE2EDuration="17.051235405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:53:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:53:35.050729211 +0000 UTC m=+3397.907813156" watchObservedRunningTime="2026-07-17 11:53:35.051235405 +0000 UTC m=+3397.908319361"
Jul 17 11:53:36.040287 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.040250    2566 generic.go:356] "Generic (PLEG): container finished" podID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerID="1719fe82e4ffd452d863b2aca9b65dd57e61fd9afac52d72f03c4d552a9332d4" exitCode=6
Jul 17 11:53:36.040736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.040321    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" event={"ID":"8478c3c8-d869-4b3e-9238-639242fdcd9f","Type":"ContainerDied","Data":"1719fe82e4ffd452d863b2aca9b65dd57e61fd9afac52d72f03c4d552a9332d4"}
Jul 17 11:53:36.040736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.040382    2566 scope.go:117] "RemoveContainer" containerID="72e32c077337cf657220745de3a78aa6de4c95189b6d81003b44c88562d2d29e"
Jul 17 11:53:36.040736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.040653    2566 scope.go:117] "RemoveContainer" containerID="1719fe82e4ffd452d863b2aca9b65dd57e61fd9afac52d72f03c4d552a9332d4"
Jul 17 11:53:36.040919 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:36.040879    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-45989be5-qvv9h_odh-ai-gateway-infra(8478c3c8-d869-4b3e-9238-639242fdcd9f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f"
Jul 17 11:53:36.067959 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.067932    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"]
Jul 17 11:53:36.154658 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.154627    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"]
Jul 17 11:53:36.159252 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.159232    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"
Jul 17 11:53:36.165579 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.165547    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"]
Jul 17 11:53:36.189941 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.189907    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc99101b-2434-4844-bf3e-56780279efa5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-bjl5l\" (UID: \"fc99101b-2434-4844-bf3e-56780279efa5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"
Jul 17 11:53:36.291145 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.291065    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc99101b-2434-4844-bf3e-56780279efa5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-bjl5l\" (UID: \"fc99101b-2434-4844-bf3e-56780279efa5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"
Jul 17 11:53:36.291691 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.291673    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc99101b-2434-4844-bf3e-56780279efa5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-bjl5l\" (UID: \"fc99101b-2434-4844-bf3e-56780279efa5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"
Jul 17 11:53:36.472050 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.472015    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"
Jul 17 11:53:36.652575 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.652540    2566 scope.go:117] "RemoveContainer" containerID="05946bcaad8203f78a62fca02032c4620a8cdfecfbc0dcc460ab708ead69b79c"
Jul 17 11:53:36.804900 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:36.804876    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"]
Jul 17 11:53:36.807936 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:53:36.807893    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc99101b_2434_4844_bf3e_56780279efa5.slice/crio-c6875b4765f63c190b0d333a7fbd012c8c14b01f648ccdfbd08ac4a67c611fa5 WatchSource:0}: Error finding container c6875b4765f63c190b0d333a7fbd012c8c14b01f648ccdfbd08ac4a67c611fa5: Status 404 returned error can't find the container with id c6875b4765f63c190b0d333a7fbd012c8c14b01f648ccdfbd08ac4a67c611fa5
Jul 17 11:53:37.055650 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:37.055585    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" event={"ID":"02b63803-4a40-48c9-867d-c5f74c389116","Type":"ContainerStarted","Data":"4b4fca91f56143ade3d40786b3c41d1cf54e35dba24c718e1946b851e844a5c7"}
Jul 17 11:53:37.056752 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:37.056728    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" event={"ID":"fc99101b-2434-4844-bf3e-56780279efa5","Type":"ContainerStarted","Data":"c6875b4765f63c190b0d333a7fbd012c8c14b01f648ccdfbd08ac4a67c611fa5"}
Jul 17 11:53:37.090202 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:37.090155    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" podStartSLOduration=18.090141136 podStartE2EDuration="18.090141136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:53:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:53:37.089852088 +0000 UTC m=+3399.946936034" watchObservedRunningTime="2026-07-17 11:53:37.090141136 +0000 UTC m=+3399.947225081"
Jul 17 11:53:37.235537 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:37.235518    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"
Jul 17 11:53:37.299620 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:37.299588    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8478c3c8-d869-4b3e-9238-639242fdcd9f-maas-api-service-ca\") pod \"8478c3c8-d869-4b3e-9238-639242fdcd9f\" (UID: \"8478c3c8-d869-4b3e-9238-639242fdcd9f\") "
Jul 17 11:53:37.299928 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:37.299907    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8478c3c8-d869-4b3e-9238-639242fdcd9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8478c3c8-d869-4b3e-9238-639242fdcd9f" (UID: "8478c3c8-d869-4b3e-9238-639242fdcd9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:53:37.400515 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:37.400484    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8478c3c8-d869-4b3e-9238-639242fdcd9f-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:53:38.063661 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.063590    2566 generic.go:356] "Generic (PLEG): container finished" podID="02b63803-4a40-48c9-867d-c5f74c389116" containerID="4b4fca91f56143ade3d40786b3c41d1cf54e35dba24c718e1946b851e844a5c7" exitCode=6
Jul 17 11:53:38.063661 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.063658    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" event={"ID":"02b63803-4a40-48c9-867d-c5f74c389116","Type":"ContainerDied","Data":"4b4fca91f56143ade3d40786b3c41d1cf54e35dba24c718e1946b851e844a5c7"}
Jul 17 11:53:38.064234 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.063737    2566 scope.go:117] "RemoveContainer" containerID="05946bcaad8203f78a62fca02032c4620a8cdfecfbc0dcc460ab708ead69b79c"
Jul 17 11:53:38.064234 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.064061    2566 scope.go:117] "RemoveContainer" containerID="4b4fca91f56143ade3d40786b3c41d1cf54e35dba24c718e1946b851e844a5c7"
Jul 17 11:53:38.064388 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:38.064302    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd_odh-ai-gateway-infra(02b63803-4a40-48c9-867d-c5f74c389116)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" podUID="02b63803-4a40-48c9-867d-c5f74c389116"
Jul 17 11:53:38.065315 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.065296    2566 generic.go:356] "Generic (PLEG): container finished" podID="fc99101b-2434-4844-bf3e-56780279efa5" containerID="280a80f5df6f1dc0b4a447566d806ed3a6ceb39cafdbe0c56f5a87cf209873e7" exitCode=6
Jul 17 11:53:38.065410 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.065328    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" event={"ID":"fc99101b-2434-4844-bf3e-56780279efa5","Type":"ContainerDied","Data":"280a80f5df6f1dc0b4a447566d806ed3a6ceb39cafdbe0c56f5a87cf209873e7"}
Jul 17 11:53:38.065547 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.065533    2566 scope.go:117] "RemoveContainer" containerID="280a80f5df6f1dc0b4a447566d806ed3a6ceb39cafdbe0c56f5a87cf209873e7"
Jul 17 11:53:38.066684 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.066662    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h" event={"ID":"8478c3c8-d869-4b3e-9238-639242fdcd9f","Type":"ContainerDied","Data":"51fc49bccbbf0c2b22363560c7d6bdcbec3fa3d0af379d4749b7572d024a7d6d"}
Jul 17 11:53:38.066879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.066687    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"
Jul 17 11:53:38.076902 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.076883    2566 scope.go:117] "RemoveContainer" containerID="1719fe82e4ffd452d863b2aca9b65dd57e61fd9afac52d72f03c4d552a9332d4"
Jul 17 11:53:38.102186 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.102158    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"]
Jul 17 11:53:38.117382 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.117355    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"]
Jul 17 11:53:38.123470 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:38.123450    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-45989be5-qvv9h"]
Jul 17 11:53:39.076529 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:39.076502    2566 generic.go:356] "Generic (PLEG): container finished" podID="fc99101b-2434-4844-bf3e-56780279efa5" containerID="8b47fecdf539d09ea218a37b78ee25219ea150822f9ed017ecca85e237e4cc04" exitCode=6
Jul 17 11:53:39.076873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:39.076543    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" event={"ID":"fc99101b-2434-4844-bf3e-56780279efa5","Type":"ContainerDied","Data":"8b47fecdf539d09ea218a37b78ee25219ea150822f9ed017ecca85e237e4cc04"}
Jul 17 11:53:39.076873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:39.076607    2566 scope.go:117] "RemoveContainer" containerID="280a80f5df6f1dc0b4a447566d806ed3a6ceb39cafdbe0c56f5a87cf209873e7"
Jul 17 11:53:39.076873 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:39.076796    2566 scope.go:117] "RemoveContainer" containerID="8b47fecdf539d09ea218a37b78ee25219ea150822f9ed017ecca85e237e4cc04"
Jul 17 11:53:39.077032 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:39.077012    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-bjl5l_odh-ai-gateway-infra(fc99101b-2434-4844-bf3e-56780279efa5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" podUID="fc99101b-2434-4844-bf3e-56780279efa5"
Jul 17 11:53:39.207719 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:39.207687    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"
Jul 17 11:53:39.318778 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:39.318738    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b63803-4a40-48c9-867d-c5f74c389116-maas-api-service-ca\") pod \"02b63803-4a40-48c9-867d-c5f74c389116\" (UID: \"02b63803-4a40-48c9-867d-c5f74c389116\") "
Jul 17 11:53:39.319070 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:39.319050    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02b63803-4a40-48c9-867d-c5f74c389116-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "02b63803-4a40-48c9-867d-c5f74c389116" (UID: "02b63803-4a40-48c9-867d-c5f74c389116"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:53:39.420174 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:39.420107    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b63803-4a40-48c9-867d-c5f74c389116-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:53:39.657896 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:39.657867    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" path="/var/lib/kubelet/pods/8478c3c8-d869-4b3e-9238-639242fdcd9f/volumes"
Jul 17 11:53:40.083601 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:40.083572    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"
Jul 17 11:53:40.084128 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:40.083567    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd" event={"ID":"02b63803-4a40-48c9-867d-c5f74c389116","Type":"ContainerDied","Data":"b43cfbbca37d92600f044f1dc5e87becca53764fa18fb42f697c01ed474f4766"}
Jul 17 11:53:40.084128 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:40.083732    2566 scope.go:117] "RemoveContainer" containerID="4b4fca91f56143ade3d40786b3c41d1cf54e35dba24c718e1946b851e844a5c7"
Jul 17 11:53:40.085547 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:40.085527    2566 scope.go:117] "RemoveContainer" containerID="8b47fecdf539d09ea218a37b78ee25219ea150822f9ed017ecca85e237e4cc04"
Jul 17 11:53:40.085805 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:40.085777    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-bjl5l_odh-ai-gateway-infra(fc99101b-2434-4844-bf3e-56780279efa5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" podUID="fc99101b-2434-4844-bf3e-56780279efa5"
Jul 17 11:53:40.101656 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:40.101630    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"]
Jul 17 11:53:40.105797 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:40.105776    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-mpnwd"]
Jul 17 11:53:41.658066 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:41.658031    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02b63803-4a40-48c9-867d-c5f74c389116" path="/var/lib/kubelet/pods/02b63803-4a40-48c9-867d-c5f74c389116/volumes"
Jul 17 11:53:44.377088 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377053    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"]
Jul 17 11:53:44.377478 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377463    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerName="revoke-keys"
Jul 17 11:53:44.377478 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377475    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerName="revoke-keys"
Jul 17 11:53:44.377548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377485    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02b63803-4a40-48c9-867d-c5f74c389116" containerName="revoke-keys"
Jul 17 11:53:44.377548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377490    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="02b63803-4a40-48c9-867d-c5f74c389116" containerName="revoke-keys"
Jul 17 11:53:44.377548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377507    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02b63803-4a40-48c9-867d-c5f74c389116" containerName="revoke-keys"
Jul 17 11:53:44.377548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377517    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="02b63803-4a40-48c9-867d-c5f74c389116" containerName="revoke-keys"
Jul 17 11:53:44.377548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377523    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02b63803-4a40-48c9-867d-c5f74c389116" containerName="revoke-keys"
Jul 17 11:53:44.377548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377528    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="02b63803-4a40-48c9-867d-c5f74c389116" containerName="revoke-keys"
Jul 17 11:53:44.377548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377545    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerName="revoke-keys"
Jul 17 11:53:44.377548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377550    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerName="revoke-keys"
Jul 17 11:53:44.377823 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377610    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerName="revoke-keys"
Jul 17 11:53:44.377823 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377617    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="02b63803-4a40-48c9-867d-c5f74c389116" containerName="revoke-keys"
Jul 17 11:53:44.377823 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377623    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="02b63803-4a40-48c9-867d-c5f74c389116" containerName="revoke-keys"
Jul 17 11:53:44.377823 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.377629    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerName="revoke-keys"
Jul 17 11:53:44.380989 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.380974    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"
Jul 17 11:53:44.387425 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.387399    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"]
Jul 17 11:53:44.463656 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.463626    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83a28ea-6a6b-43ca-96cf-c811deb74188-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d\" (UID: \"e83a28ea-6a6b-43ca-96cf-c811deb74188\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"
Jul 17 11:53:44.564777 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.564747    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83a28ea-6a6b-43ca-96cf-c811deb74188-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d\" (UID: \"e83a28ea-6a6b-43ca-96cf-c811deb74188\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"
Jul 17 11:53:44.565356 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.565336    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83a28ea-6a6b-43ca-96cf-c811deb74188-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d\" (UID: \"e83a28ea-6a6b-43ca-96cf-c811deb74188\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"
Jul 17 11:53:44.693514 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.693441    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"
Jul 17 11:53:44.824668 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:53:44.824640    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode83a28ea_6a6b_43ca_96cf_c811deb74188.slice/crio-c2582e5f7d3dc3c3c3ea64b52c05c19d9028f80894248912020b73d574b6ffd4 WatchSource:0}: Error finding container c2582e5f7d3dc3c3c3ea64b52c05c19d9028f80894248912020b73d574b6ffd4: Status 404 returned error can't find the container with id c2582e5f7d3dc3c3c3ea64b52c05c19d9028f80894248912020b73d574b6ffd4
Jul 17 11:53:44.824826 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:44.824745    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"]
Jul 17 11:53:45.112960 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:45.112936    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d" event={"ID":"e83a28ea-6a6b-43ca-96cf-c811deb74188","Type":"ContainerStarted","Data":"8fc8e546b0196f1e5c4ec7042d54ff11041414138324b89b20e0a5b25b8630d6"}
Jul 17 11:53:45.113081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:45.112983    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d" event={"ID":"e83a28ea-6a6b-43ca-96cf-c811deb74188","Type":"ContainerStarted","Data":"c2582e5f7d3dc3c3c3ea64b52c05c19d9028f80894248912020b73d574b6ffd4"}
Jul 17 11:53:45.133863 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:45.133809    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d" podStartSLOduration=1.13379192 podStartE2EDuration="1.13379192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:53:45.130673904 +0000 UTC m=+3407.987757866" watchObservedRunningTime="2026-07-17 11:53:45.13379192 +0000 UTC m=+3407.990875866"
Jul 17 11:53:46.119312 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:46.119278    2566 generic.go:356] "Generic (PLEG): container finished" podID="e83a28ea-6a6b-43ca-96cf-c811deb74188" containerID="8fc8e546b0196f1e5c4ec7042d54ff11041414138324b89b20e0a5b25b8630d6" exitCode=0
Jul 17 11:53:46.119700 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:46.119320    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d" event={"ID":"e83a28ea-6a6b-43ca-96cf-c811deb74188","Type":"ContainerDied","Data":"8fc8e546b0196f1e5c4ec7042d54ff11041414138324b89b20e0a5b25b8630d6"}
Jul 17 11:53:47.259177 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:47.259157    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"
Jul 17 11:53:47.389629 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:47.389551    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83a28ea-6a6b-43ca-96cf-c811deb74188-maas-api-service-ca\") pod \"e83a28ea-6a6b-43ca-96cf-c811deb74188\" (UID: \"e83a28ea-6a6b-43ca-96cf-c811deb74188\") "
Jul 17 11:53:47.389834 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:47.389804    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e83a28ea-6a6b-43ca-96cf-c811deb74188-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e83a28ea-6a6b-43ca-96cf-c811deb74188" (UID: "e83a28ea-6a6b-43ca-96cf-c811deb74188"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:53:47.389953 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:47.389909    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83a28ea-6a6b-43ca-96cf-c811deb74188-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:53:47.660260 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:47.657103    2566 scope.go:117] "RemoveContainer" containerID="10139e4c3080b30c7442abaee203fee90609f47b1b8b110311b6c5862b67f31b"
Jul 17 11:53:48.131853 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:48.131811    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" event={"ID":"5dd459aa-5b85-42a1-aea1-dfb15326de5c","Type":"ContainerStarted","Data":"1286fbc51a360832d71ef04a9f86f678c9d29db4b1cb21ade690c233333161f1"}
Jul 17 11:53:48.133264 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:48.133187    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d"
Jul 17 11:53:48.133264 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:48.133232    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d" event={"ID":"e83a28ea-6a6b-43ca-96cf-c811deb74188","Type":"ContainerDied","Data":"c2582e5f7d3dc3c3c3ea64b52c05c19d9028f80894248912020b73d574b6ffd4"}
Jul 17 11:53:48.133460 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:48.133290    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2582e5f7d3dc3c3c3ea64b52c05c19d9028f80894248912020b73d574b6ffd4"
Jul 17 11:53:48.151301 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:48.151258    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" podStartSLOduration=19.151244324 podStartE2EDuration="19.151244324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:53:48.14878806 +0000 UTC m=+3411.005872027" watchObservedRunningTime="2026-07-17 11:53:48.151244324 +0000 UTC m=+3411.008328269"
Jul 17 11:53:49.140879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:49.140846    2566 generic.go:356] "Generic (PLEG): container finished" podID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerID="1286fbc51a360832d71ef04a9f86f678c9d29db4b1cb21ade690c233333161f1" exitCode=6
Jul 17 11:53:49.141358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:49.140924    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" event={"ID":"5dd459aa-5b85-42a1-aea1-dfb15326de5c","Type":"ContainerDied","Data":"1286fbc51a360832d71ef04a9f86f678c9d29db4b1cb21ade690c233333161f1"}
Jul 17 11:53:49.141358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:49.141003    2566 scope.go:117] "RemoveContainer" containerID="10139e4c3080b30c7442abaee203fee90609f47b1b8b110311b6c5862b67f31b"
Jul 17 11:53:49.141358 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:49.141308    2566 scope.go:117] "RemoveContainer" containerID="1286fbc51a360832d71ef04a9f86f678c9d29db4b1cb21ade690c233333161f1"
Jul 17 11:53:49.141519 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:49.141500    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj_odh-ai-gateway-infra(5dd459aa-5b85-42a1-aea1-dfb15326de5c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c"
Jul 17 11:53:49.159281 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:49.159260    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"]
Jul 17 11:53:50.285225 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:50.285201    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"
Jul 17 11:53:50.417614 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:50.417532    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd459aa-5b85-42a1-aea1-dfb15326de5c-maas-api-service-ca\") pod \"5dd459aa-5b85-42a1-aea1-dfb15326de5c\" (UID: \"5dd459aa-5b85-42a1-aea1-dfb15326de5c\") "
Jul 17 11:53:50.417887 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:50.417864    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd459aa-5b85-42a1-aea1-dfb15326de5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5dd459aa-5b85-42a1-aea1-dfb15326de5c" (UID: "5dd459aa-5b85-42a1-aea1-dfb15326de5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:53:50.518129 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:50.518099    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd459aa-5b85-42a1-aea1-dfb15326de5c-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:53:51.154339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:51.154304    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"
Jul 17 11:53:51.154339 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:51.154327    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj" event={"ID":"5dd459aa-5b85-42a1-aea1-dfb15326de5c","Type":"ContainerDied","Data":"77cb6f1cf0bc675cfb84fb58c573b3e5240d060383426195de880d59297332ca"}
Jul 17 11:53:51.154576 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:51.154384    2566 scope.go:117] "RemoveContainer" containerID="1286fbc51a360832d71ef04a9f86f678c9d29db4b1cb21ade690c233333161f1"
Jul 17 11:53:51.181655 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:51.181628    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"]
Jul 17 11:53:51.188091 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:51.188069    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-4xwtj"]
Jul 17 11:53:51.651723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:51.651674    2566 scope.go:117] "RemoveContainer" containerID="8b47fecdf539d09ea218a37b78ee25219ea150822f9ed017ecca85e237e4cc04"
Jul 17 11:53:51.657633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:51.657605    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" path="/var/lib/kubelet/pods/5dd459aa-5b85-42a1-aea1-dfb15326de5c/volumes"
Jul 17 11:53:52.162086 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:52.162052    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" event={"ID":"fc99101b-2434-4844-bf3e-56780279efa5","Type":"ContainerStarted","Data":"fa4532814e8fb0198747e8c64ca588dcc29f823477f75d3a9ce693bd2eb50de1"}
Jul 17 11:53:52.181145 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:52.181095    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" podStartSLOduration=16.181080771 podStartE2EDuration="16.181080771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:53:52.179806354 +0000 UTC m=+3415.036890312" watchObservedRunningTime="2026-07-17 11:53:52.181080771 +0000 UTC m=+3415.038164715"
Jul 17 11:53:53.169276 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:53.169241    2566 generic.go:356] "Generic (PLEG): container finished" podID="fc99101b-2434-4844-bf3e-56780279efa5" containerID="fa4532814e8fb0198747e8c64ca588dcc29f823477f75d3a9ce693bd2eb50de1" exitCode=6
Jul 17 11:53:53.169756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:53.169310    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" event={"ID":"fc99101b-2434-4844-bf3e-56780279efa5","Type":"ContainerDied","Data":"fa4532814e8fb0198747e8c64ca588dcc29f823477f75d3a9ce693bd2eb50de1"}
Jul 17 11:53:53.169756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:53.169369    2566 scope.go:117] "RemoveContainer" containerID="8b47fecdf539d09ea218a37b78ee25219ea150822f9ed017ecca85e237e4cc04"
Jul 17 11:53:53.169756 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:53.169616    2566 scope.go:117] "RemoveContainer" containerID="fa4532814e8fb0198747e8c64ca588dcc29f823477f75d3a9ce693bd2eb50de1"
Jul 17 11:53:53.169907 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:53:53.169852    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-bjl5l_odh-ai-gateway-infra(fc99101b-2434-4844-bf3e-56780279efa5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" podUID="fc99101b-2434-4844-bf3e-56780279efa5"
Jul 17 11:53:53.197978 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:53.197956    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"]
Jul 17 11:53:54.313121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:54.313099    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"
Jul 17 11:53:54.451688 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:54.451604    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc99101b-2434-4844-bf3e-56780279efa5-maas-api-service-ca\") pod \"fc99101b-2434-4844-bf3e-56780279efa5\" (UID: \"fc99101b-2434-4844-bf3e-56780279efa5\") "
Jul 17 11:53:54.452006 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:54.451982    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc99101b-2434-4844-bf3e-56780279efa5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc99101b-2434-4844-bf3e-56780279efa5" (UID: "fc99101b-2434-4844-bf3e-56780279efa5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:53:54.552448 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:54.552410    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc99101b-2434-4844-bf3e-56780279efa5-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:53:55.183004 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:55.182971    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l" event={"ID":"fc99101b-2434-4844-bf3e-56780279efa5","Type":"ContainerDied","Data":"c6875b4765f63c190b0d333a7fbd012c8c14b01f648ccdfbd08ac4a67c611fa5"}
Jul 17 11:53:55.183004 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:55.182992    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"
Jul 17 11:53:55.183221 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:55.183031    2566 scope.go:117] "RemoveContainer" containerID="fa4532814e8fb0198747e8c64ca588dcc29f823477f75d3a9ce693bd2eb50de1"
Jul 17 11:53:55.208816 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:55.208784    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"]
Jul 17 11:53:55.214596 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:55.214575    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-bjl5l"]
Jul 17 11:53:55.658415 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:55.658382    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc99101b-2434-4844-bf3e-56780279efa5" path="/var/lib/kubelet/pods/fc99101b-2434-4844-bf3e-56780279efa5/volumes"
Jul 17 11:53:58.245390 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:53:58.245355    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nclqh"]
Jul 17 11:54:10.211534 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:10.211498    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"]
Jul 17 11:54:10.212008 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:10.211747    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy" containerID="cri-o://dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366" gracePeriod=30
Jul 17 11:54:14.400648 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:14.400602    2566 patch_prober.go:28] interesting pod/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 11:54:14.401072 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:14.400669    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 11:54:15.462995 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.462974    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:54:15.543810 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.543739    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cktxd\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-kube-api-access-cktxd\") pod \"36e4e997-bfb1-417a-a59f-bc38972a15ad\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") "
Jul 17 11:54:15.543810 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.543789    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-credential-socket\") pod \"36e4e997-bfb1-417a-a59f-bc38972a15ad\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") "
Jul 17 11:54:15.544030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.543822    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-token\") pod \"36e4e997-bfb1-417a-a59f-bc38972a15ad\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") "
Jul 17 11:54:15.544030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.543850    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e4e997-bfb1-417a-a59f-bc38972a15ad-istiod-ca-cert\") pod \"36e4e997-bfb1-417a-a59f-bc38972a15ad\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") "
Jul 17 11:54:15.544030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.543867    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-certs\") pod \"36e4e997-bfb1-417a-a59f-bc38972a15ad\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") "
Jul 17 11:54:15.544030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.543888    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-envoy\") pod \"36e4e997-bfb1-417a-a59f-bc38972a15ad\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") "
Jul 17 11:54:15.544030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.543924    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-podinfo\") pod \"36e4e997-bfb1-417a-a59f-bc38972a15ad\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") "
Jul 17 11:54:15.544030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.543958    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-socket\") pod \"36e4e997-bfb1-417a-a59f-bc38972a15ad\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") "
Jul 17 11:54:15.544030 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.543986    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-data\") pod \"36e4e997-bfb1-417a-a59f-bc38972a15ad\" (UID: \"36e4e997-bfb1-417a-a59f-bc38972a15ad\") "
Jul 17 11:54:15.544388 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.544265    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36e4e997-bfb1-417a-a59f-bc38972a15ad" (UID: "36e4e997-bfb1-417a-a59f-bc38972a15ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:54:15.544388 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.544282    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36e4e997-bfb1-417a-a59f-bc38972a15ad" (UID: "36e4e997-bfb1-417a-a59f-bc38972a15ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:54:15.544388 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.544319    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e4e997-bfb1-417a-a59f-bc38972a15ad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36e4e997-bfb1-417a-a59f-bc38972a15ad" (UID: "36e4e997-bfb1-417a-a59f-bc38972a15ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:54:15.544556 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.544508    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36e4e997-bfb1-417a-a59f-bc38972a15ad" (UID: "36e4e997-bfb1-417a-a59f-bc38972a15ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:54:15.544660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.544615    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-credential-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:15.544660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.544634    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e4e997-bfb1-417a-a59f-bc38972a15ad-istiod-ca-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:15.544660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.544648    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:15.544824 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.544661    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-workload-socket\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:15.544927 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.544898    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-data" (OuterVolumeSpecName: "istio-data") pod "36e4e997-bfb1-417a-a59f-bc38972a15ad" (UID: "36e4e997-bfb1-417a-a59f-bc38972a15ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:54:15.546035 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.546006    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-kube-api-access-cktxd" (OuterVolumeSpecName: "kube-api-access-cktxd") pod "36e4e997-bfb1-417a-a59f-bc38972a15ad" (UID: "36e4e997-bfb1-417a-a59f-bc38972a15ad"). InnerVolumeSpecName "kube-api-access-cktxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:54:15.546117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.546072    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36e4e997-bfb1-417a-a59f-bc38972a15ad" (UID: "36e4e997-bfb1-417a-a59f-bc38972a15ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 11:54:15.546361 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.546347    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-token" (OuterVolumeSpecName: "istio-token") pod "36e4e997-bfb1-417a-a59f-bc38972a15ad" (UID: "36e4e997-bfb1-417a-a59f-bc38972a15ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 11:54:15.546456 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.546438    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36e4e997-bfb1-417a-a59f-bc38972a15ad" (UID: "36e4e997-bfb1-417a-a59f-bc38972a15ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 11:54:15.645315 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.645287    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cktxd\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-kube-api-access-cktxd\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:15.645315 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.645314    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-token\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:15.645466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.645324    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-envoy\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:15.645466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.645333    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-podinfo\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:15.645466 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:15.645340    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e4e997-bfb1-417a-a59f-bc38972a15ad-istio-data\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:16.301929 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.301894    2566 generic.go:356] "Generic (PLEG): container finished" podID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerID="dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366" exitCode=0
Jul 17 11:54:16.302110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.301947    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" event={"ID":"36e4e997-bfb1-417a-a59f-bc38972a15ad","Type":"ContainerDied","Data":"dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366"}
Jul 17 11:54:16.302110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.301951    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"
Jul 17 11:54:16.302110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.302016    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl" event={"ID":"36e4e997-bfb1-417a-a59f-bc38972a15ad","Type":"ContainerDied","Data":"addeda30d17795793f11328040d1ef417942f4d587e576cdda556c9305d13ed0"}
Jul 17 11:54:16.302110 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.302039    2566 scope.go:117] "RemoveContainer" containerID="dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366"
Jul 17 11:54:16.310991 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.310970    2566 scope.go:117] "RemoveContainer" containerID="f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd"
Jul 17 11:54:16.318668 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.318650    2566 scope.go:117] "RemoveContainer" containerID="dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366"
Jul 17 11:54:16.318947 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:16.318927    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366\": container with ID starting with dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366 not found: ID does not exist" containerID="dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366"
Jul 17 11:54:16.319007 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.318954    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366"} err="failed to get container status \"dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366\": rpc error: code = NotFound desc = could not find container \"dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366\": container with ID starting with dc94b25d8f440bbf6a2e9a2d59583aee460085effe3be8c6efcd4da1ecf69366 not found: ID does not exist"
Jul 17 11:54:16.319007 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.318976    2566 scope.go:117] "RemoveContainer" containerID="f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd"
Jul 17 11:54:16.319209 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:16.319191    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd\": container with ID starting with f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd not found: ID does not exist" containerID="f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd"
Jul 17 11:54:16.319264 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.319218    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd"} err="failed to get container status \"f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd\": rpc error: code = NotFound desc = could not find container \"f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd\": container with ID starting with f3d7264026357e108dd63a86778bdc571f5b3f7363a95800ca2368d4c0d070dd not found: ID does not exist"
Jul 17 11:54:16.323308 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.323285    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"]
Jul 17 11:54:16.330735 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.330698    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-9d836f-openshift-default-5476fb7b9f-zprnl"]
Jul 17 11:54:16.567546 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.567474    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jvkbh_c37e45b5-fbaa-4fb9-a521-223df2aba955/manager/0.log"
Jul 17 11:54:16.695927 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.695895    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-srkz4_2aaade70-96a8-4214-b6a7-db0ebd7bec93/manager/0.log"
Jul 17 11:54:16.961401 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:16.961374    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-btbtp_d434868f-84a2-4de3-9f5f-1f706be96803/server/0.log"
Jul 17 11:54:17.341514 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:17.341488    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd97f7fc7-b22vs_41bfc339-8743-485e-ab59-004f8dffca27/copy-manifests/0.log"
Jul 17 11:54:17.355607 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:17.355572    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd97f7fc7-b22vs_41bfc339-8743-485e-ab59-004f8dffca27/manager/0.log"
Jul 17 11:54:17.657855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:17.657776    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" path="/var/lib/kubelet/pods/36e4e997-bfb1-417a-a59f-bc38972a15ad/volumes"
Jul 17 11:54:17.981523 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:17.981445    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738115-xvs6r_93d45de2-506a-4b15-b0be-0ee8c281c36f/cleanup/0.log"
Jul 17 11:54:18.093423 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:18.093398    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738130-v6v8k_ea7a9d80-bdcf-4158-a2aa-a913c9168672/cleanup/0.log"
Jul 17 11:54:18.212633 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:18.212599    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738145-42ntj_20cb8118-63a1-4a98-a625-91442c941d88/cleanup/0.log"
Jul 17 11:54:18.668317 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:18.668283    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-fc3287-q4fsv_cbaa2e44-d2ed-4c68-b0e0-b5ceaa941986/revoke-keys/0.log"
Jul 17 11:54:18.798693 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:18.798663    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-7c5b42-x59q8_00c62ac0-771c-49c4-b542-6aad30b89b57/revoke-keys/0.log"
Jul 17 11:54:18.915550 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:18.915523    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-5a6d7b05-ctc85_3f9322e6-d2c9-46e4-a964-f1902ec2d66c/revoke-keys/0.log"
Jul 17 11:54:19.141938 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.141914    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-ad4893-zf667_1337966a-dabf-439d-9ff6-0b3c54842a80/revoke-keys/0.log"
Jul 17 11:54:19.254284 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.254264    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-9d836f-wcf2d_e83a28ea-6a6b-43ca-96cf-c811deb74188/revoke-keys/0.log"
Jul 17 11:54:19.378180 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378147    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"]
Jul 17 11:54:19.378537 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378524    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy"
Jul 17 11:54:19.378586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378539    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy"
Jul 17 11:54:19.378586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378547    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerName="revoke-keys"
Jul 17 11:54:19.378586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378553    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerName="revoke-keys"
Jul 17 11:54:19.378586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378563    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy"
Jul 17 11:54:19.378586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378569    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy"
Jul 17 11:54:19.378586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378580    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerName="revoke-keys"
Jul 17 11:54:19.378586 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378585    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378596    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc99101b-2434-4844-bf3e-56780279efa5" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378601    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc99101b-2434-4844-bf3e-56780279efa5" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378607    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e83a28ea-6a6b-43ca-96cf-c811deb74188" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378612    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e83a28ea-6a6b-43ca-96cf-c811deb74188" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378623    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378628    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378634    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc99101b-2434-4844-bf3e-56780279efa5" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378639    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc99101b-2434-4844-bf3e-56780279efa5" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378690    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e83a28ea-6a6b-43ca-96cf-c811deb74188" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378699    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="02b63803-4a40-48c9-867d-c5f74c389116" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378720    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378725    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378732    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8478c3c8-d869-4b3e-9238-639242fdcd9f" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378737    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc99101b-2434-4844-bf3e-56780279efa5" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378743    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e4e997-bfb1-417a-a59f-bc38972a15ad" containerName="istio-proxy"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378749    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc99101b-2434-4844-bf3e-56780279efa5" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378754    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerName="revoke-keys"
Jul 17 11:54:19.378812 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.378758    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc99101b-2434-4844-bf3e-56780279efa5" containerName="revoke-keys"
Jul 17 11:54:19.383501 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.383483    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"
Jul 17 11:54:19.386377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.386361    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 11:54:19.387914 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.387890    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cwv2h\""
Jul 17 11:54:19.389995 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.389973    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"]
Jul 17 11:54:19.479032 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.478962    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8486f54f-e206-4426-812c-f0df4447392f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9\" (UID: \"8486f54f-e206-4426-812c-f0df4447392f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"
Jul 17 11:54:19.579578 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.579542    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8486f54f-e206-4426-812c-f0df4447392f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9\" (UID: \"8486f54f-e206-4426-812c-f0df4447392f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"
Jul 17 11:54:19.580169 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.580148    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8486f54f-e206-4426-812c-f0df4447392f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9\" (UID: \"8486f54f-e206-4426-812c-f0df4447392f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"
Jul 17 11:54:19.695232 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.695205    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"
Jul 17 11:54:19.845238 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:19.845210    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"]
Jul 17 11:54:19.847112 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:54:19.847088    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8486f54f_e206_4426_812c_f0df4447392f.slice/crio-9869eef74942aee8431ce7e9f1d034a1d75a0cc4087274c85ac9a20ad9a6cb47 WatchSource:0}: Error finding container 9869eef74942aee8431ce7e9f1d034a1d75a0cc4087274c85ac9a20ad9a6cb47: Status 404 returned error can't find the container with id 9869eef74942aee8431ce7e9f1d034a1d75a0cc4087274c85ac9a20ad9a6cb47
Jul 17 11:54:20.324453 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:20.324424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" event={"ID":"8486f54f-e206-4426-812c-f0df4447392f","Type":"ContainerStarted","Data":"d83e2047bc5cebb6a219ebb6d4c7043b2c98fe265675336f81282a0a1bfc12a4"}
Jul 17 11:54:20.324655 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:20.324471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" event={"ID":"8486f54f-e206-4426-812c-f0df4447392f","Type":"ContainerStarted","Data":"9869eef74942aee8431ce7e9f1d034a1d75a0cc4087274c85ac9a20ad9a6cb47"}
Jul 17 11:54:20.345000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:20.344946    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" podStartSLOduration=1.34493187 podStartE2EDuration="1.34493187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:54:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:54:20.342788492 +0000 UTC m=+3443.199872436" watchObservedRunningTime="2026-07-17 11:54:20.34493187 +0000 UTC m=+3443.202015814"
Jul 17 11:54:21.330910 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:21.330883    2566 generic.go:356] "Generic (PLEG): container finished" podID="8486f54f-e206-4426-812c-f0df4447392f" containerID="d83e2047bc5cebb6a219ebb6d4c7043b2c98fe265675336f81282a0a1bfc12a4" exitCode=6
Jul 17 11:54:21.331377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:21.330971    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" event={"ID":"8486f54f-e206-4426-812c-f0df4447392f","Type":"ContainerDied","Data":"d83e2047bc5cebb6a219ebb6d4c7043b2c98fe265675336f81282a0a1bfc12a4"}
Jul 17 11:54:21.331377 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:21.331109    2566 scope.go:117] "RemoveContainer" containerID="d83e2047bc5cebb6a219ebb6d4c7043b2c98fe265675336f81282a0a1bfc12a4"
Jul 17 11:54:21.459336 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:21.459309    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nclqh_00908a0f-f8d4-42c7-a1c6-6014d807b6f9/limitador/0.log"
Jul 17 11:54:21.584300 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:21.584235    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-d4knq_d3784d55-7352-4002-a85a-d40559e24d64/manager/0.log"
Jul 17 11:54:22.078303 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:22.078277    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nntqq_b07e7bfa-cef9-4293-a26b-e12149e7042b/discovery/0.log"
Jul 17 11:54:22.195154 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:22.195070    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c556c8c74-47nl9_693bb079-731e-47ea-b59e-fb9846c28d3d/kube-auth-proxy/0.log"
Jul 17 11:54:22.338394 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:22.338363    2566 generic.go:356] "Generic (PLEG): container finished" podID="8486f54f-e206-4426-812c-f0df4447392f" containerID="df3a0cea50c305e2ad2572a98bb30b89b4a70ce31d65a5eddd25b5ec2b20932f" exitCode=6
Jul 17 11:54:22.338844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:22.338430    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" event={"ID":"8486f54f-e206-4426-812c-f0df4447392f","Type":"ContainerDied","Data":"df3a0cea50c305e2ad2572a98bb30b89b4a70ce31d65a5eddd25b5ec2b20932f"}
Jul 17 11:54:22.338844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:22.338477    2566 scope.go:117] "RemoveContainer" containerID="d83e2047bc5cebb6a219ebb6d4c7043b2c98fe265675336f81282a0a1bfc12a4"
Jul 17 11:54:22.338844 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:22.338693    2566 scope.go:117] "RemoveContainer" containerID="df3a0cea50c305e2ad2572a98bb30b89b4a70ce31d65a5eddd25b5ec2b20932f"
Jul 17 11:54:22.339026 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:22.338951    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9_odh-ai-gateway-infra(8486f54f-e206-4426-812c-f0df4447392f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" podUID="8486f54f-e206-4426-812c-f0df4447392f"
Jul 17 11:54:22.434854 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:22.434811    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hkwn8_93486233-9ab9-4859-ba3e-711e53d4c05a/istio-proxy/1.log"
Jul 17 11:54:23.345736 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:23.345695    2566 scope.go:117] "RemoveContainer" containerID="df3a0cea50c305e2ad2572a98bb30b89b4a70ce31d65a5eddd25b5ec2b20932f"
Jul 17 11:54:23.346154 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:23.345904    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9_odh-ai-gateway-infra(8486f54f-e206-4426-812c-f0df4447392f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" podUID="8486f54f-e206-4426-812c-f0df4447392f"
Jul 17 11:54:23.412226 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:23.412199    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2c4mf_8690b803-7fc8-4a56-914e-7c0dc674275b/main/0.log"
Jul 17 11:54:26.207635 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.207600    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"]
Jul 17 11:54:26.208121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.207962    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerName="revoke-keys"
Jul 17 11:54:26.208121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.207974    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerName="revoke-keys"
Jul 17 11:54:26.208121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.207993    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc99101b-2434-4844-bf3e-56780279efa5" containerName="revoke-keys"
Jul 17 11:54:26.208121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.207999    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc99101b-2434-4844-bf3e-56780279efa5" containerName="revoke-keys"
Jul 17 11:54:26.208121 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.208055    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd459aa-5b85-42a1-aea1-dfb15326de5c" containerName="revoke-keys"
Jul 17 11:54:26.212026 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.212008    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"
Jul 17 11:54:26.219975 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.219946    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"]
Jul 17 11:54:26.237824 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.237799    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-2cq66\" (UID: \"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"
Jul 17 11:54:26.339077 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.339046    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-2cq66\" (UID: \"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"
Jul 17 11:54:26.339649 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.339630    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57741a73-2cq66\" (UID: \"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"
Jul 17 11:54:26.523356 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.523274    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"
Jul 17 11:54:26.655515 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:26.655490    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"]
Jul 17 11:54:26.657103 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:54:26.657077    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f3a8f9_e4c5_4f5c_92a2_7ab206577e97.slice/crio-b2385faaa58fc9732cc0d7735fc65c53c4623cca55f64d67c897dfa1dec7021c WatchSource:0}: Error finding container b2385faaa58fc9732cc0d7735fc65c53c4623cca55f64d67c897dfa1dec7021c: Status 404 returned error can't find the container with id b2385faaa58fc9732cc0d7735fc65c53c4623cca55f64d67c897dfa1dec7021c
Jul 17 11:54:27.367175 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:27.367147    2566 generic.go:356] "Generic (PLEG): container finished" podID="b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97" containerID="4d752ecb292841929e3fdf6cbe14205458b2f9f85a51db3792f3bb961ebb4898" exitCode=6
Jul 17 11:54:27.367501 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:27.367229    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66" event={"ID":"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97","Type":"ContainerDied","Data":"4d752ecb292841929e3fdf6cbe14205458b2f9f85a51db3792f3bb961ebb4898"}
Jul 17 11:54:27.367501 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:27.367280    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66" event={"ID":"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97","Type":"ContainerStarted","Data":"b2385faaa58fc9732cc0d7735fc65c53c4623cca55f64d67c897dfa1dec7021c"}
Jul 17 11:54:27.367501 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:27.367391    2566 scope.go:117] "RemoveContainer" containerID="4d752ecb292841929e3fdf6cbe14205458b2f9f85a51db3792f3bb961ebb4898"
Jul 17 11:54:28.373998 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:28.373966    2566 generic.go:356] "Generic (PLEG): container finished" podID="b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97" containerID="627c31b58934cd8584d6e61b5ce4033932f0ddbebe4fbdcf7d065c7e1a320a31" exitCode=6
Jul 17 11:54:28.374400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:28.374057    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66" event={"ID":"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97","Type":"ContainerDied","Data":"627c31b58934cd8584d6e61b5ce4033932f0ddbebe4fbdcf7d065c7e1a320a31"}
Jul 17 11:54:28.374400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:28.374115    2566 scope.go:117] "RemoveContainer" containerID="4d752ecb292841929e3fdf6cbe14205458b2f9f85a51db3792f3bb961ebb4898"
Jul 17 11:54:28.374400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:28.374242    2566 scope.go:117] "RemoveContainer" containerID="627c31b58934cd8584d6e61b5ce4033932f0ddbebe4fbdcf7d065c7e1a320a31"
Jul 17 11:54:28.374532 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:28.374499    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-2cq66_odh-ai-gateway-infra(b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66" podUID="b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97"
Jul 17 11:54:29.381488 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:29.381462    2566 scope.go:117] "RemoveContainer" containerID="627c31b58934cd8584d6e61b5ce4033932f0ddbebe4fbdcf7d065c7e1a320a31"
Jul 17 11:54:29.381864 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:29.381630    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-2cq66_odh-ai-gateway-infra(b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66" podUID="b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97"
Jul 17 11:54:29.531648 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:29.531615    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"]
Jul 17 11:54:29.536115 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:29.536098    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"
Jul 17 11:54:29.544133 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:29.544111    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"]
Jul 17 11:54:29.568397 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:29.568370    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b82427d-74f9-4e79-a8e3-39879c618436-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d\" (UID: \"7b82427d-74f9-4e79-a8e3-39879c618436\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"
Jul 17 11:54:29.669323 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:29.669246    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b82427d-74f9-4e79-a8e3-39879c618436-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d\" (UID: \"7b82427d-74f9-4e79-a8e3-39879c618436\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"
Jul 17 11:54:29.669906 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:29.669888    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b82427d-74f9-4e79-a8e3-39879c618436-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d\" (UID: \"7b82427d-74f9-4e79-a8e3-39879c618436\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"
Jul 17 11:54:29.847046 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:29.847015    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"
Jul 17 11:54:29.986509 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:29.986480    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"]
Jul 17 11:54:29.988485 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:54:29.988457    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b82427d_74f9_4e79_a8e3_39879c618436.slice/crio-617ce26e43fa27dfc5b6eea80f9350046e596cf4c1bb89fef16f92649c8e27ac WatchSource:0}: Error finding container 617ce26e43fa27dfc5b6eea80f9350046e596cf4c1bb89fef16f92649c8e27ac: Status 404 returned error can't find the container with id 617ce26e43fa27dfc5b6eea80f9350046e596cf4c1bb89fef16f92649c8e27ac
Jul 17 11:54:30.387892 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:30.387860    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" event={"ID":"7b82427d-74f9-4e79-a8e3-39879c618436","Type":"ContainerStarted","Data":"517f0515f64476c4cd22f1b10fe82e505b20d4fa0f19361f2ef5b77a5502382a"}
Jul 17 11:54:30.388289 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:30.387917    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" event={"ID":"7b82427d-74f9-4e79-a8e3-39879c618436","Type":"ContainerStarted","Data":"617ce26e43fa27dfc5b6eea80f9350046e596cf4c1bb89fef16f92649c8e27ac"}
Jul 17 11:54:30.406237 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:30.406192    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" podStartSLOduration=1.406179311 podStartE2EDuration="1.406179311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:54:30.404771805 +0000 UTC m=+3453.261855750" watchObservedRunningTime="2026-07-17 11:54:30.406179311 +0000 UTC m=+3453.263263255"
Jul 17 11:54:31.394319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:31.394239    2566 generic.go:356] "Generic (PLEG): container finished" podID="7b82427d-74f9-4e79-a8e3-39879c618436" containerID="517f0515f64476c4cd22f1b10fe82e505b20d4fa0f19361f2ef5b77a5502382a" exitCode=6
Jul 17 11:54:31.394319 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:31.394271    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" event={"ID":"7b82427d-74f9-4e79-a8e3-39879c618436","Type":"ContainerDied","Data":"517f0515f64476c4cd22f1b10fe82e505b20d4fa0f19361f2ef5b77a5502382a"}
Jul 17 11:54:31.394743 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:31.394494    2566 scope.go:117] "RemoveContainer" containerID="517f0515f64476c4cd22f1b10fe82e505b20d4fa0f19361f2ef5b77a5502382a"
Jul 17 11:54:32.401251 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:32.401164    2566 generic.go:356] "Generic (PLEG): container finished" podID="7b82427d-74f9-4e79-a8e3-39879c618436" containerID="6aa6b90c834a63df93805a763fb4139b1b638252f61e7202abbfe7f77b0a7b0e" exitCode=6
Jul 17 11:54:32.401251 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:32.401211    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" event={"ID":"7b82427d-74f9-4e79-a8e3-39879c618436","Type":"ContainerDied","Data":"6aa6b90c834a63df93805a763fb4139b1b638252f61e7202abbfe7f77b0a7b0e"}
Jul 17 11:54:32.401773 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:32.401259    2566 scope.go:117] "RemoveContainer" containerID="517f0515f64476c4cd22f1b10fe82e505b20d4fa0f19361f2ef5b77a5502382a"
Jul 17 11:54:32.401773 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:32.401477    2566 scope.go:117] "RemoveContainer" containerID="6aa6b90c834a63df93805a763fb4139b1b638252f61e7202abbfe7f77b0a7b0e"
Jul 17 11:54:32.401773 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:32.401687    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d_odh-ai-gateway-infra(7b82427d-74f9-4e79-a8e3-39879c618436)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" podUID="7b82427d-74f9-4e79-a8e3-39879c618436"
Jul 17 11:54:32.953698 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:32.953664    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8kq7q_0758dbfc-ac21-4268-83e7-1cb5053415b8/global-pull-secret-syncer/0.log"
Jul 17 11:54:33.149295 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:33.149258    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q2wd4_4df32018-f34a-4db9-b449-1c6dddd2d6db/konnectivity-agent/0.log"
Jul 17 11:54:33.242444 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:33.242357    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-138.ec2.internal_47af943b4b7e55afd17e31a4e269c392/haproxy/0.log"
Jul 17 11:54:33.410895 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:33.410863    2566 scope.go:117] "RemoveContainer" containerID="6aa6b90c834a63df93805a763fb4139b1b638252f61e7202abbfe7f77b0a7b0e"
Jul 17 11:54:33.411307 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:33.411284    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d_odh-ai-gateway-infra(7b82427d-74f9-4e79-a8e3-39879c618436)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" podUID="7b82427d-74f9-4e79-a8e3-39879c618436"
Jul 17 11:54:38.556452 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:38.556424    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nclqh_00908a0f-f8d4-42c7-a1c6-6014d807b6f9/limitador/0.log"
Jul 17 11:54:38.630528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:38.630493    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-d4knq_d3784d55-7352-4002-a85a-d40559e24d64/manager/0.log"
Jul 17 11:54:38.651483 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:38.651462    2566 scope.go:117] "RemoveContainer" containerID="df3a0cea50c305e2ad2572a98bb30b89b4a70ce31d65a5eddd25b5ec2b20932f"
Jul 17 11:54:39.446780 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:39.446748    2566 generic.go:356] "Generic (PLEG): container finished" podID="8486f54f-e206-4426-812c-f0df4447392f" containerID="745521f12f9877f5787148a2668ed85319bb018d64f1d1cdb642c1749536b671" exitCode=6
Jul 17 11:54:39.446938 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:39.446821    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" event={"ID":"8486f54f-e206-4426-812c-f0df4447392f","Type":"ContainerDied","Data":"745521f12f9877f5787148a2668ed85319bb018d64f1d1cdb642c1749536b671"}
Jul 17 11:54:39.446938 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:39.446906    2566 scope.go:117] "RemoveContainer" containerID="df3a0cea50c305e2ad2572a98bb30b89b4a70ce31d65a5eddd25b5ec2b20932f"
Jul 17 11:54:39.447185 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:39.447170    2566 scope.go:117] "RemoveContainer" containerID="745521f12f9877f5787148a2668ed85319bb018d64f1d1cdb642c1749536b671"
Jul 17 11:54:39.447380 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:39.447358    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9_odh-ai-gateway-infra(8486f54f-e206-4426-812c-f0df4447392f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" podUID="8486f54f-e206-4426-812c-f0df4447392f"
Jul 17 11:54:40.396510 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.396485    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b8ab507-0b7d-407a-8cfd-c72368a0ba02/alertmanager/0.log"
Jul 17 11:54:40.430602 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.430577    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b8ab507-0b7d-407a-8cfd-c72368a0ba02/config-reloader/0.log"
Jul 17 11:54:40.467032 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.467009    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b8ab507-0b7d-407a-8cfd-c72368a0ba02/kube-rbac-proxy-web/0.log"
Jul 17 11:54:40.473855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.473835    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"]
Jul 17 11:54:40.495764 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.495736    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b8ab507-0b7d-407a-8cfd-c72368a0ba02/kube-rbac-proxy/0.log"
Jul 17 11:54:40.524723 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.524685    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b8ab507-0b7d-407a-8cfd-c72368a0ba02/kube-rbac-proxy-metric/0.log"
Jul 17 11:54:40.558937 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.558906    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b8ab507-0b7d-407a-8cfd-c72368a0ba02/prom-label-proxy/0.log"
Jul 17 11:54:40.595307 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.595277    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6b8ab507-0b7d-407a-8cfd-c72368a0ba02/init-config-reloader/0.log"
Jul 17 11:54:40.630410 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.630384    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"
Jul 17 11:54:40.764312 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.764279    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8486f54f-e206-4426-812c-f0df4447392f-maas-api-service-ca\") pod \"8486f54f-e206-4426-812c-f0df4447392f\" (UID: \"8486f54f-e206-4426-812c-f0df4447392f\") "
Jul 17 11:54:40.764621 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.764599    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8486f54f-e206-4426-812c-f0df4447392f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8486f54f-e206-4426-812c-f0df4447392f" (UID: "8486f54f-e206-4426-812c-f0df4447392f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:54:40.855104 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.855020    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-5fkfh_a9d66ff0-9de6-46f8-8742-5e66bf07d950/monitoring-plugin/0.log"
Jul 17 11:54:40.865557 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.865537    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8486f54f-e206-4426-812c-f0df4447392f-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:40.891788 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.891764    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8jmst_dfb33b7f-1020-45f8-b3a2-b7b23102e2d9/node-exporter/0.log"
Jul 17 11:54:40.921246 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.921230    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8jmst_dfb33b7f-1020-45f8-b3a2-b7b23102e2d9/kube-rbac-proxy/0.log"
Jul 17 11:54:40.952330 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:40.952310    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8jmst_dfb33b7f-1020-45f8-b3a2-b7b23102e2d9/init-textfile/0.log"
Jul 17 11:54:41.459793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:41.459763    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"
Jul 17 11:54:41.460213 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:41.459813    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9" event={"ID":"8486f54f-e206-4426-812c-f0df4447392f","Type":"ContainerDied","Data":"9869eef74942aee8431ce7e9f1d034a1d75a0cc4087274c85ac9a20ad9a6cb47"}
Jul 17 11:54:41.460213 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:41.459872    2566 scope.go:117] "RemoveContainer" containerID="745521f12f9877f5787148a2668ed85319bb018d64f1d1cdb642c1749536b671"
Jul 17 11:54:41.485948 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:41.485927    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"]
Jul 17 11:54:41.492400 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:41.492379    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40b693f1-c9bn9"]
Jul 17 11:54:41.557931 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:41.557908    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55dffc6f4d-gd845_0367d82f-a576-4753-bf69-d536f4a8cfad/telemeter-client/0.log"
Jul 17 11:54:41.583651 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:41.583627    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55dffc6f4d-gd845_0367d82f-a576-4753-bf69-d536f4a8cfad/reload/0.log"
Jul 17 11:54:41.608788 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:41.608769    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55dffc6f4d-gd845_0367d82f-a576-4753-bf69-d536f4a8cfad/kube-rbac-proxy/0.log"
Jul 17 11:54:41.657966 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:41.657942    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8486f54f-e206-4426-812c-f0df4447392f" path="/var/lib/kubelet/pods/8486f54f-e206-4426-812c-f0df4447392f/volumes"
Jul 17 11:54:43.016418 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.016391    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-889sw_e7d3da95-5ede-4a1f-bd2f-119ca091ecdd/networking-console-plugin/0.log"
Jul 17 11:54:43.450139 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.450107    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"]
Jul 17 11:54:43.450612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.450592    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8486f54f-e206-4426-812c-f0df4447392f" containerName="revoke-keys"
Jul 17 11:54:43.450612 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.450613    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8486f54f-e206-4426-812c-f0df4447392f" containerName="revoke-keys"
Jul 17 11:54:43.450817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.450647    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8486f54f-e206-4426-812c-f0df4447392f" containerName="revoke-keys"
Jul 17 11:54:43.450817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.450655    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8486f54f-e206-4426-812c-f0df4447392f" containerName="revoke-keys"
Jul 17 11:54:43.450817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.450758    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8486f54f-e206-4426-812c-f0df4447392f" containerName="revoke-keys"
Jul 17 11:54:43.450817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.450776    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8486f54f-e206-4426-812c-f0df4447392f" containerName="revoke-keys"
Jul 17 11:54:43.453906 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.453886    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.456551 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.456527    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lgfmw\"/\"kube-root-ca.crt\""
Jul 17 11:54:43.456765 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.456752    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lgfmw\"/\"default-dockercfg-tk7jl\""
Jul 17 11:54:43.457793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.457762    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lgfmw\"/\"openshift-service-ca.crt\""
Jul 17 11:54:43.462689 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.462668    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"]
Jul 17 11:54:43.594126 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.594092    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-podres\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.594294 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.594140    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vl9n\" (UniqueName: \"kubernetes.io/projected/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-kube-api-access-4vl9n\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.594294 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.594204    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-lib-modules\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.594294 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.594270    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-proc\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.594401 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.594333    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-sys\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.651765 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.651733    2566 scope.go:117] "RemoveContainer" containerID="627c31b58934cd8584d6e61b5ce4033932f0ddbebe4fbdcf7d065c7e1a320a31"
Jul 17 11:54:43.695774 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.695750    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-lib-modules\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.695855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.695802    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-proc\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.695855 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.695837    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-sys\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.695940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.695873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-podres\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.695940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.695914    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vl9n\" (UniqueName: \"kubernetes.io/projected/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-kube-api-access-4vl9n\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.695940 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.695927    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-lib-modules\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.696049 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.695927    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-proc\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.696049 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.695951    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-sys\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.696049 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.696011    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-podres\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.706003 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.705956    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vl9n\" (UniqueName: \"kubernetes.io/projected/e2e0811b-1b57-4c30-8a88-2c33e2f83e51-kube-api-access-4vl9n\") pod \"perf-node-gather-daemonset-mnsqd\" (UID: \"e2e0811b-1b57-4c30-8a88-2c33e2f83e51\") " pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.756507 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.756483    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/1.log"
Jul 17 11:54:43.761108 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.761084    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mf2tf_57e7a55d-d9b5-4406-aa5c-7ce1b40ee65d/console-operator/2.log"
Jul 17 11:54:43.764321 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.764307    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:43.892895 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:43.892872    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"]
Jul 17 11:54:43.894577 ip-10-0-134-138 kubenswrapper[2566]: W0717 11:54:43.894548    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode2e0811b_1b57_4c30_8a88_2c33e2f83e51.slice/crio-93dc484fcaa8d66564296a125ca3f10a5da55faa919eab92b5d43c8c194eb21c WatchSource:0}: Error finding container 93dc484fcaa8d66564296a125ca3f10a5da55faa919eab92b5d43c8c194eb21c: Status 404 returned error can't find the container with id 93dc484fcaa8d66564296a125ca3f10a5da55faa919eab92b5d43c8c194eb21c
Jul 17 11:54:44.376771 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.376749    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6587cf4f48-wj762_3e2d8d4e-31b8-49cd-81b9-fd3a6c5dc78a/console/0.log"
Jul 17 11:54:44.478298 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.478269    2566 generic.go:356] "Generic (PLEG): container finished" podID="b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97" containerID="c71fa886fa3f407bd71de15aeb4395b4f7662e5836d381f954c087c44ba218bf" exitCode=6
Jul 17 11:54:44.478482 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.478341    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66" event={"ID":"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97","Type":"ContainerDied","Data":"c71fa886fa3f407bd71de15aeb4395b4f7662e5836d381f954c087c44ba218bf"}
Jul 17 11:54:44.478482 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.478409    2566 scope.go:117] "RemoveContainer" containerID="627c31b58934cd8584d6e61b5ce4033932f0ddbebe4fbdcf7d065c7e1a320a31"
Jul 17 11:54:44.478779 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.478763    2566 scope.go:117] "RemoveContainer" containerID="c71fa886fa3f407bd71de15aeb4395b4f7662e5836d381f954c087c44ba218bf"
Jul 17 11:54:44.479015 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:44.478994    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-57741a73-2cq66_odh-ai-gateway-infra(b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66" podUID="b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97"
Jul 17 11:54:44.479876 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.479857    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd" event={"ID":"e2e0811b-1b57-4c30-8a88-2c33e2f83e51","Type":"ContainerStarted","Data":"e18965c494fb12c1330b760cb3c576e2873c5cdec635c5036321d21aafbc6788"}
Jul 17 11:54:44.479978 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.479894    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd" event={"ID":"e2e0811b-1b57-4c30-8a88-2c33e2f83e51","Type":"ContainerStarted","Data":"93dc484fcaa8d66564296a125ca3f10a5da55faa919eab92b5d43c8c194eb21c"}
Jul 17 11:54:44.480036 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.479984    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:44.513281 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.513235    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd" podStartSLOduration=1.513222656 podStartE2EDuration="1.513222656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 11:54:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:54:44.510794141 +0000 UTC m=+3467.367878086" watchObservedRunningTime="2026-07-17 11:54:44.513222656 +0000 UTC m=+3467.370306592"
Jul 17 11:54:44.652067 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:44.651998    2566 scope.go:117] "RemoveContainer" containerID="6aa6b90c834a63df93805a763fb4139b1b638252f61e7202abbfe7f77b0a7b0e"
Jul 17 11:54:45.487207 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:45.487128    2566 generic.go:356] "Generic (PLEG): container finished" podID="7b82427d-74f9-4e79-a8e3-39879c618436" containerID="03247c1dc26ed4949923603c7be9c1a1eeeacf4ae3ccac5145854d90e2ada52e" exitCode=6
Jul 17 11:54:45.487572 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:45.487203    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" event={"ID":"7b82427d-74f9-4e79-a8e3-39879c618436","Type":"ContainerDied","Data":"03247c1dc26ed4949923603c7be9c1a1eeeacf4ae3ccac5145854d90e2ada52e"}
Jul 17 11:54:45.487572 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:45.487295    2566 scope.go:117] "RemoveContainer" containerID="6aa6b90c834a63df93805a763fb4139b1b638252f61e7202abbfe7f77b0a7b0e"
Jul 17 11:54:45.487660 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:45.487638    2566 scope.go:117] "RemoveContainer" containerID="03247c1dc26ed4949923603c7be9c1a1eeeacf4ae3ccac5145854d90e2ada52e"
Jul 17 11:54:45.487886 ip-10-0-134-138 kubenswrapper[2566]: E0717 11:54:45.487858    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d_odh-ai-gateway-infra(7b82427d-74f9-4e79-a8e3-39879c618436)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" podUID="7b82427d-74f9-4e79-a8e3-39879c618436"
Jul 17 11:54:45.507529 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:45.507498    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"]
Jul 17 11:54:45.648211 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:45.648188    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"
Jul 17 11:54:45.815166 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:45.815095    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97-maas-api-service-ca\") pod \"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97\" (UID: \"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97\") "
Jul 17 11:54:45.815444 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:45.815422    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97" (UID: "b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:54:45.916640 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:45.916614    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:46.211240 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.211214    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7hgrl_679aabef-721d-436b-977a-87f5d0608638/dns/0.log"
Jul 17 11:54:46.234973 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.234953    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7hgrl_679aabef-721d-436b-977a-87f5d0608638/kube-rbac-proxy/0.log"
Jul 17 11:54:46.430404 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.430375    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rxwbd_6aefeda7-ce38-4020-8d0b-6ea66c2d3d36/dns-node-resolver/0.log"
Jul 17 11:54:46.493957 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.493877    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"
Jul 17 11:54:46.494393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.493879    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66" event={"ID":"b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97","Type":"ContainerDied","Data":"b2385faaa58fc9732cc0d7735fc65c53c4623cca55f64d67c897dfa1dec7021c"}
Jul 17 11:54:46.494393 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.494018    2566 scope.go:117] "RemoveContainer" containerID="c71fa886fa3f407bd71de15aeb4395b4f7662e5836d381f954c087c44ba218bf"
Jul 17 11:54:46.518651 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.518627    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"]
Jul 17 11:54:46.521306 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.521287    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"]
Jul 17 11:54:46.527000 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.526981    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57741a73-2cq66"]
Jul 17 11:54:46.658528 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.658507    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"
Jul 17 11:54:46.824026 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.824003    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b82427d-74f9-4e79-a8e3-39879c618436-maas-api-service-ca\") pod \"7b82427d-74f9-4e79-a8e3-39879c618436\" (UID: \"7b82427d-74f9-4e79-a8e3-39879c618436\") "
Jul 17 11:54:46.824329 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.824305    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b82427d-74f9-4e79-a8e3-39879c618436-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b82427d-74f9-4e79-a8e3-39879c618436" (UID: "7b82427d-74f9-4e79-a8e3-39879c618436"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 11:54:46.925006 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:46.924978    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b82427d-74f9-4e79-a8e3-39879c618436-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 17 11:54:47.081818 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:47.081745    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pf6hp_cfea99fc-e552-4e24-a93b-8bfe7a6ddb5e/node-ca/0.log"
Jul 17 11:54:47.502879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:47.502805    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"
Jul 17 11:54:47.502879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:47.502856    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d" event={"ID":"7b82427d-74f9-4e79-a8e3-39879c618436","Type":"ContainerDied","Data":"617ce26e43fa27dfc5b6eea80f9350046e596cf4c1bb89fef16f92649c8e27ac"}
Jul 17 11:54:47.503350 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:47.502909    2566 scope.go:117] "RemoveContainer" containerID="03247c1dc26ed4949923603c7be9c1a1eeeacf4ae3ccac5145854d90e2ada52e"
Jul 17 11:54:47.525879 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:47.525853    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"]
Jul 17 11:54:47.534642 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:47.534615    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-829f05b9-vdd4d"]
Jul 17 11:54:47.658290 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:47.658259    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b82427d-74f9-4e79-a8e3-39879c618436" path="/var/lib/kubelet/pods/7b82427d-74f9-4e79-a8e3-39879c618436/volumes"
Jul 17 11:54:47.658592 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:47.658580    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97" path="/var/lib/kubelet/pods/b4f3a8f9-e4c5-4f5c-92a2-7ab206577e97/volumes"
Jul 17 11:54:48.267928 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:48.267900    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nntqq_b07e7bfa-cef9-4293-a26b-e12149e7042b/discovery/0.log"
Jul 17 11:54:48.292857 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:48.292797    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c556c8c74-47nl9_693bb079-731e-47ea-b59e-fb9846c28d3d/kube-auth-proxy/0.log"
Jul 17 11:54:48.356788 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:48.356759    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hkwn8_93486233-9ab9-4859-ba3e-711e53d4c05a/istio-proxy/0.log"
Jul 17 11:54:48.393790 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:48.393769    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hkwn8_93486233-9ab9-4859-ba3e-711e53d4c05a/istio-proxy/1.log"
Jul 17 11:54:49.261350 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:49.261319    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-47rdw_289bbdfa-ebd8-42af-be5d-95b7702a39f4/serve-healthcheck-canary/0.log"
Jul 17 11:54:50.020970 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:50.020941    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rgk7n_ff832f52-dc1e-4629-9ba0-8f872f0ecbf4/kube-rbac-proxy/0.log"
Jul 17 11:54:50.046541 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:50.046514    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rgk7n_ff832f52-dc1e-4629-9ba0-8f872f0ecbf4/exporter/0.log"
Jul 17 11:54:50.075909 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:50.075885    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rgk7n_ff832f52-dc1e-4629-9ba0-8f872f0ecbf4/extractor/0.log"
Jul 17 11:54:50.497793 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:50.497763    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lgfmw/perf-node-gather-daemonset-mnsqd"
Jul 17 11:54:52.398081 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:52.397993    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jvkbh_c37e45b5-fbaa-4fb9-a521-223df2aba955/manager/0.log"
Jul 17 11:54:52.545987 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:52.545958    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-srkz4_2aaade70-96a8-4214-b6a7-db0ebd7bec93/manager/0.log"
Jul 17 11:54:52.728813 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:52.728790    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-btbtp_d434868f-84a2-4de3-9f5f-1f706be96803/server/0.log"
Jul 17 11:54:52.976817 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:52.976746    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd97f7fc7-b22vs_41bfc339-8743-485e-ab59-004f8dffca27/manager/0.log"
Jul 17 11:54:52.997942 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:52.997919    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd97f7fc7-b22vs_41bfc339-8743-485e-ab59-004f8dffca27/copy-manifests/0.log"
Jul 17 11:54:54.750464 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:54.750431    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nntqq_b07e7bfa-cef9-4293-a26b-e12149e7042b/discovery/0.log"
Jul 17 11:54:54.762264 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:54.762237    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c556c8c74-47nl9_693bb079-731e-47ea-b59e-fb9846c28d3d/kube-auth-proxy/0.log"
Jul 17 11:54:54.806382 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:54.806360    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hkwn8_93486233-9ab9-4859-ba3e-711e53d4c05a/istio-proxy/0.log"
Jul 17 11:54:54.842918 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:54.842894    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hkwn8_93486233-9ab9-4859-ba3e-711e53d4c05a/istio-proxy/1.log"
Jul 17 11:54:56.292725 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:56.292687    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nntqq_b07e7bfa-cef9-4293-a26b-e12149e7042b/discovery/0.log"
Jul 17 11:54:56.305232 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:56.305209    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c556c8c74-47nl9_693bb079-731e-47ea-b59e-fb9846c28d3d/kube-auth-proxy/0.log"
Jul 17 11:54:56.350494 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:56.350469    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hkwn8_93486233-9ab9-4859-ba3e-711e53d4c05a/istio-proxy/0.log"
Jul 17 11:54:56.379548 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:56.379524    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hkwn8_93486233-9ab9-4859-ba3e-711e53d4c05a/istio-proxy/1.log"
Jul 17 11:54:59.897630 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:59.897599    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-hmd6d_d5fe56a1-f8b0-4583-ab74-817cb77a6a2f/migrator/0.log"
Jul 17 11:54:59.921117 ip-10-0-134-138 kubenswrapper[2566]: I0717 11:54:59.921086    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-hmd6d_d5fe56a1-f8b0-4583-ab74-817cb77a6a2f/graceful-termination/0.log"